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ABSTRACT' 

This report reviews the education research and 
development funding policies of the National Institute of Education 
(NIE) and attempts to evaluate the impact of those policies on the 
nation's education research and development system, particularly the 
regional education laboratories and research and development centers 
established by the Federal government in the 1960s. The report is 
based on the work of ten consultants and represents a synthesis of 
the different consultants" views. An additional paper prepared by one 
of the consultants is included as an appendix. Individual chapters cf 
the report discuss the task and procedures of the consultants, the 
role of research and development in improving education, present 
resources for education research and development, the context for 
policy-making at NIE, policy directions at NIE, and the conclusions 
and recommendations of the consultants. (JG) 
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PREFACE 



This report is based on the work of ten consultants to the 
National Institute of Education and the National Council on Educational 
Research. The group came together at the request of the Acting Director 
of the Institute and the National Council in March 1975, to review edu- 
cation research and development (R&D) funding policies now in effect 
and proposed for the near future. The review was for the purpose of 
evaluating the impact of such policies on the nation's education R&D 
system, with special reference to the regional education laboratories 
and research and development centers established by the government in 
the 1960s. 

This was a substantial charge, and various data-collection and 
analysis aspects of it merit extensive continuing attention within 
the Institute, as we suggest in our recommendations. More time than 
the three months available to the consultants would have permitted a 
more comprehensive survey of NIE's funding policies and possibly more 
accurate predictions of their consequences. We have attempted to 
relate the policy directions we see to certain data available on 
research and development in education, and to other information we 
gathered. But in the end, we have relied on the judgment of the 
consultants to interpret the probable impact of present policies and 
to make recommendations for improvement. 

A good many people have helped us in the course of our inquiry. 
Staff at a number of education R&D organizations, including several 
regional laboratories and research and development centers, took time 
to confer with us during visits. Sixteen labs and centers generously 
provided extensive and candid written responses to questions we posed. 
Several dozen knowledgeable people across the country shared with us 
in writing their own analyses of present policies in education R&D 
funding. The Acting Director of NIE and many of his staff met with 
us, graciously accommodating our requests for early luorning or evening 
hours beyond the call of duty. The New York State Department of 
Education handled the printing, distribution, and receipt of a ques- 
tionnaire quickly and skillfully, for which we thank our consultant 
colleague Ewald Nyquist. 

I am pleased to acknowledge also the consultants' appreciation 
of the contributions made by our Executive Secretary, Frederick (Fritz) 
Mulhauser, and his associate, Maureen Ttfeacy. Without their excellent 
technical assistance, the work of our group would have been much more 
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difficult. The support of other staff in the Dissemination and 
Resources Group at NIE is also acknowledged. The consultants alone, 
of course, are responsible for the content and style of this report. 

As a group, we find ourselves stronger NIE supporters now than 
when began. Even so, there are a number of steps NIE can take 
which in our view will make its funding policies more effective and 
we have tried to speak frankly about those steps. The report which 
follows represents a synthesis of the views of ten consultants. 

From the beginning it was agreed that any consultant might 
provide an individual statement which would supplement or differ 
from the synthesis of the others' views. Sam D. Sieber gave us one 
such paper, on the design requirements of the research and development 
system, which we include as Appendix A. 

I am most grateful to my colleagues for their patient reviews of 
drafts of this report. 

We appreciate the opportunity to examine the funding policies of 
the Institute, and the openness of its leaders, staff, and contractors 
to our inquiry. We trust our conclusions and recommendations will be 
found useful to the Director, the National Council, and others con- 
cerned with improving education through research and development. 



Roald F. Campbell 
Principal Consultant 



I CHARGE TO THE CONSULTANTS 
AND PROCEDURES 



In a memorandum to the National Council on Educational Research 
dated April 4, 1975, Acting NIE Director Emerson Elliott outlined the 
purpose of the consultants' work. We were to give our advice to both 
the Institute and the Council about "alternative policies which the 
Institute might adopt for funding education R&D activities." Within 
this extremely broad general mandate, we were to give "special atten- 
tion" to the regional educational laboratories and research and 
development centers established by the Federal government in the 
last decade. Elliott further amplified the charge: 

This advice will be based on a review of NIE funding policies 
and their effects on various R&D performers and will further 
be based on the consultants' assessment of: 

(1) the existing capacity within the nation for high quality 
educational research and development; 

(2) the past relationship of various federal funding policies 
to the maintenance and improvement of research and develop- 
ment capacity — to meet both current and projected needs; 

(3) the potential effects of alternative NIE funding policies 
on the maintenance and improvement of such capacity; and 

(4) the relative advantages and risks of principal alternative 
funding policies. 

The memorandum explained part of the motivation for the study by 
reviewing the history of scarce funds and sharply focussed programs 
within the NIE, and the resulting likelihood that some previously- 
supported R&D institutions "will not receive NIE funds equal to past 
levels of Federal support (i.e., their capacity will be underutilized 
by NIE)... The question then arises whether capacity not now being util- 
ized by NIE and other funding sources under their current priorities is 
of such value to the education community that it needs to be preserved 
through extraordinary efforts." Mr. Elliott also stressed to the Council 
the Institute's concern to establish and maintain an effective education 
"R&D system" as called for by the authorizing law, and the Institute's 
need for advice on the effects of its policies on the system. He noted 
the particular concerns of the regional laboratories and R&D centers, 
which believe that the current Institute plans for Fiscal Year (FY) 1976 
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will in some ways damage their capabilities to the detriment of the 
nation's R&D effort. 

For the consultants, Roald Campbell attempted at an April 4 
meeting with the Council to make clear that the charge was very broad 
and that the time available would permit judgment and advice based 
chiefly on experience and general knowledge, rather than extensive 
fresh survey of the state of the nation's education R&D. The Council 
joined the Acting Director in expressing support for the consultants' 
work. 

To carry out the charge, the ten consultants determined to use 
the following procedures: 

1. Meet in person with NIE management, program staff, and 
selected projrxt officers; commission a review by staff 
of present NIE policies and their context; solicit in 
writing the views of NIE staff. 

2. Meet with laboratory and R&D center directors in person, 
make visits to selected institutions, and gather further 
information from all labs and centers by a questionnaire. 

3. Meet in person with representatives of groups which have 
interests in education R&D, including teachers, researchers, 
teacher-educators, administrators. Congressional staff, and 
staff of other Federal agencies. 

4. Solicit in writing the views of individuals knowledgeable 
about R&D and related areas. 

5. Visit selected R&D-per forming institutions other than 
Federally-created laboratories and centers, and others that 
were at one tiine laboratories and centers, but have ceased 
to have major Federal support. 

6. Review selected literature on education R&D generally, and 
the Federal role in the field. 

Two weeks after the initial meeting with the NIE Director and 
Council on April 4, the group met with representatives of the regional 
laboratories, R&D centers, and the Executive Director of the Council for 
Educational Development and Research (CEDaR) at Chicago, April 18. At 
that same time, the consultants reviewed the staff paper on the present 
policies of the Institute, and the context of policy-making. In the 
following month, to begin assessing the state of present capacity for 
R&D, consultants visited eleven R&D institutions, wrote to over 50 
persons in the field, and arranged a variety of meetings with individ- 
uals and groups. During the same period, staff prepared memoranda on 
various subjects requested, and circulated relevant literature. 
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To gather up-to-date information on the regional laboratories apd 
R&D centers and on the likely impact of NIE policies there, the con- 
sultants drafted a questionnaire and reviewed it with the lab/center 
group at Chicago. After further modification, ten questions were 
posed to sixteen institutions. Confidential responses to one sensitive 
question on strengths and weaknesses of past relations with the NIE 
were channeled directly to one consultant; the rest were analyzed by 
staff. (The questionnaire itself, and summaries of responses to certain 
questions, are in Appendices B and C.) 

For a third meeting on May 22-24, consultants exchanged written 
reports on meetings and visits, and circulated replies to correspondence 
and questionnaires. Based on extensive discussion of findings, a draft 
of consultants' views was prepared for a fourth meeting at New York, 
June 5-6. Further conclusions and recommendations were considered at 
that time, resulting in additional drafts circulated for review 
and comment. 

To gather information about the Institute, the consultants met twice 
with the Acting Director, and with the newly-appointed Director. In 
addition, consultants were briefed about each of the Institute programs 
by Associate Directors, and one consultant interviewed seven project 
officers from different program areas to understand policies at the 
working level. Consultants also asked each program group to rate cur- 
rent work under way at laboratories and centers, as to its quality and 
relevance to the mission of the unit. 

Appendix D includes a complete list of places visited, persons and 
groups met with, and correspondence received. 

The procedures described have given the consultants a three-month 
glimpse at the present state of education R&D, including the NIE and 
certain performers within that system. Literature such as the draft 
1975'Databook on the status of education R&D in the United States has 
added a larger perspective."^ Nevertheless, as stated in the initial 
meetings with the Director and the National Council, the discussion, 
conclusions, and recommendations which follow are based on the consult- 
ants' general experience and contact with Federal agencies, universities, 
research and development organizations, state education agencies, and 
local school systems, as much as on new data collected. 

Our report is organized into four sections* of analysis and dis- 



W. Paisley and associates. The Status of Educational R esearch and 
Development in the United States: 1975 Databook (Washington, D.C.: 
National Institute of Education; pre-publication version. May 1975). 
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cussions and a final section of conclusions and recommendations. 
The chapters grow increasingly focussed, moving from consideration 
of the role of research generally in education and the resources 
available for R&D, in Chapters II and III, to discussion of the 
NIE context and policy in Chapters IV and V. Conclusions drawn 
from the discussion in all four chapters, and specific recommendations 
which derive from them, are presented in Chapter VI, 
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II THE ROLE OF R&D IN 
IMPROVING EDUCATION 



A broad climate of expectation about research generally, and social 
science research part'icularly , affects policy-making within the NIE. 
Hence we feel it is useful to begin with some attention to more general 
aspects of the role of research and development in improving education. 
In this section, we note some current voices of pessimism, offer some 
explanation of inevitable difficulties facing R&D in human service 
fields, and end with our own sense of the usefulness of the activity 
of disciplined inquiry. 

An observer in the last few years could note a wave of criticism 
and uncertainty about research: 

® A recent monograph sponsored by Ralph Nader attacks the 
objectivity and quality of work by the nation's most pres- 
tigious scientific body, the National Academy of Sciences. 

^ A vote of the House of Representatives directs the National 
Science Foundation to submit every proposed grant of funds 
for Congressional review prior to award. 

Members of Congress have expressed strong concern over the 
social science R&D contracting procedures throughout the 
Department of HEW, and over the support of social studies 
curriculum work and other social research at the National 
Science Foundation. 

^ Congress is even beginning to question some health research, 
as continued billions invested particularly in cancer studies 
seem to have little- immediate result. 

And at the same time as these events at the Federal level concern- 
ing research generally, one could also note questions raised in the edu- 
cation community about the value of educational R&D. Decision-makers 
are said to be pressing for "hard evidence" that new products will "de- 
liver'' Che results that policy boards are see. ing through accountability 
schemes. Staff within education, as in other human services, may be 
coming to view the results of research as chiefly a series of complications 
to their professional lives: recommendations for uprooting the structures 
and personal relationships that once gave security within the basic working 
units or organizations or even wholesale condemnations of the present system 
and the urging of its rebuilding in some other form. Such "R&D results" 
do not call forth much support from teachers or other workers in the field. 

For ourselves, I we have noted some characteristics of inquiry in 

11 
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the social or human sciences and characteristics of the education 
system the inquiry is aimed at helping, which perhaps can give per- 
spective on both the inflated hopes of the 1960s and the pessimism 
of the mid-1970s. 

First, in all human service fields, education included, the de- 
mand for purely service funds always exceeds the available resources. 
As a result, money spent for inquiry and development of ideas will 
always be suspect, as a competitor against supplying "the real needs" 
of children, the elderly, the handicapped, or others. Thus, we expect 
an inevitable need for justification, explanation, and attention to 
the politics of funding. Oversight—by Congress or other funding body— 
of how research funds will be spent, and pressure against such spending, 
are thus not a temporary condition, but a permanent fact of life. 

Second, the American public education system is not controlled 
from a single point, and is subject to influence from the full 
spectrum of social forces — fads. and popular whims, court decisions 
and evolving legal philosophies, incentives created by shifts in mar- 
ginal funds available from state or Federal sources, and the decisions 
of thousands of atomistic actors — clients, providers, and controllers 
of education. Furthermore, in a system so vast as education, even 
similar units (schools, districts, colleges, state agencies) are differ- 
ent from each other, and are changing and developing at varying rates, 
under varying pressures. The system characteristics of openness, vul- 
nerability, and complexity have implications for inquiry ac^'lvity: 

^ knowledge will inevitably be tentative, as the syste^u is 
constantly changing, and even is altered by inquiry into 
it; 

^ knowledge will not automatically be universally applicable, 
even to all like-named pieces of the system ("schools," etc.); 

^ inquiry will inevitably be seen as a weak tool in a process 
of change and improvement, as so many other forces press on 
the system, particularly now the forces of legislation, court 
action, and tight money. 

A third limitation on the us e f ulnes s of inquiry i s tha t even a t 
its most successful, the results will not be self-executing. Actors 
in the educational system have free will to some degree, and have 
diverse preferences which they seek to carry out. Thus a research 
finding that certain activities by the teacher can reliably produce 
certain results in a classroom will require a long chain of deliberate 
action to produce results — starting with the question of goals 
or ends, "Does anyone want to attain the results we can now reliably 
produce?" 
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Fourth, another consequence of the complexity of the process 
being studied in education research is that an enormous range of kinds 
of inquiry are relevant. We must accept the fact that any one study 
or line of research, at any particular time, will yield few "break- 
throughs." Education involves the physiology of eye and brain, the 
psychology of attention, memory, and learning, the sociology of peer, 
group, and organizational interaction, and so on. Can any one research 
activity be faulted for not unlocking the whole puzzle, as was literally 
possible on some areas of physical science not so long ago? 

These four features of inquiry in education — combine to give us 
restrained expectations for the role of research and development in 
improving schools. Both Congressional and professional observers 
of education R&D need to adopt realistic hopes for short-run impacts 
of the enterprise. We all must moderate the impulse to rip up 
the structure and activity every year or two if it seems not to 
be delivering rapid improvement. The common observation of the 
"failure" of social programs of the 1960s should not be allowed to 
lead to hasty pressure for catching up in areas of missing knowledge. 
A crash program and accpmpanying inflated goals is as unrealistic 
in the area of knowledge-production and utilization as in any 
of the service programs of an earlier era against poverty or other 
problems. ^ 

For the National Institute of Education, we conclude that there 
is a need for the Institute to speak out more forcefully, to lead 
the debate and to express its own goals rather than primarily 
reacting to others. The National Council on Educational Research, 
it seems to us, has made a good start in talking about the 
reauthorization of NIE. Their minutes show a concern that the 
NIE not be expected to have solved the problems of education, but 
rather to have placed a variety of useful ideas and products 
into channels from which educators can elect to use them. But 
with a new Director, and a full- strength Council in the near 
future, it will be essential to do a great deal more explaining of the 
grounds on which the NIE wishes to be viewed — through highly- 
visible statements and more extensive publications and communications 
with the field and the Congress. 
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We note familiar rhetoric used recently in a news story to describe 
another "attack" and the subsequent disillusionment. '*War on Cancer 
Stirs Political Baclclash," New York Times , May 27, 1975, p.l. 
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Having urged that expectations be clarified, we reaffirm com- 
pletely the wisdom of Congress and the Executive Branch in estab- 
lishing the Institute, as an important means 

can education. Though the Institute's life has been brief, and its 
critics legion, its future potential is fully equal to the Institutes 
of Health or the Science Foundation, and its stature should be equiva- 
lent. We find the untapped possibilities so great, even within moderate 
expectations of results both in the Institute's direct support of re- 
search and as x^ell in a coordinating and planning role for a wide 
range of public and private inquiry in education, that we urge long- 
term authorization. Further, though we recognize the current 
tight budget, we urge both the Administration and the Congress 
not to unduly narrow the future authorization of funds. The 
annual spending ceiling of $80 million in the Administration reauthor- 
ization bilL is far too low, and should be doubled in the first year 
alone. The appropriations process provides a good check on the actual 
expenditure plans; we see no need to set in the authorizing statute 
such a restrictive limit to the potential activity of the Institute. 

We conclude that there are inevitable limits to the role of in- 
quiry as a tool of educational change, and that these need to be 
recognized far more than they have been by the Institute, its publics, 
and the Congress. Yet we also judge that the limits have hardly been 
reached, and that the legal and financial authority of the Institute 
should be commensurate with the potential and the challenge. In the 
next section, we turn to consideration of the resources available to 
carry out disciplined inquiry into education. 
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Ill THE PRESENT RESOURCES 
FOR EDUCATION R&D 



Placing the NIE policies and their impact in a context has 
been a key element of our work. Our charge from the first has 
included some attempt to assess the "capacity" for high quality 
research and development in education in the country, against which 
to test present and proposed NIE policy directions. This capacity 
includes scientific and technical personnel in diverse roles of 
researchers, developers. It includes the. institutions which 
house these individuals, and their varied patterns of structure, 
incentive, and performance. And capacity depends also on the coher 
ence or system-quality of the aggregate, the degree to which the 
whole is greater than the sum. of the parts. We begin with some 
general impressions we have formed about the development of education 
R&D to this point, then ^Iscuss in more detail some concerns we have 
about the present supply of skilled people in the field, and the forms 
so far used to arrange them. We conclude the section with our assess- 
ment of the relatively weak integration or inter-relation of the over- 
all system. 

The consultants have been impressed with a number of features 
of the expansion of the education R&D effort in this country in the 
last two decades: 

1. There has been an impressive history of growth of the 
resources for education R&D, including growth in size 
and sophistication: 

° increases in funds invested (a trend until recently); 

° wider range of talent involved in such work; 

,° new people added to the resources through training; ^ 

° new institutions formed or drawn into education R&D 
owing to new resources and interest; 

° emergence of new technologies and specializations 
such as education research management, systematic 
product development, and evaluation. 

2. There has been a striking continuity of Federal support for 
education R&D across five very different administrations - 
from the passage of the Cooperative Research Act in 1954 to 
the present, leading from the first funds for research, to 
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creation of new external institutions in the laboratories 
and research centers, and finally to creation of an entirely 
new Federal asencv. the mtt?. 



3. There appears to be an increasing volume of research in- 
formation and educational products available year by year— 
though not always readilv Wnni^ a^^^oo-!Ki„ .^.1 , 
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igh not always readily known, accessible, or even in 
demand. 

There is steadily increasing sophistication of study and 
discourse through the work of social scientists and others 
who look beyond the classroom, survey experts who can pro- 
vide very large samples, analytic techniques to permit 
large-scale generalizations and evaluation studies of a 
decade of local and Federal initiatives which can be treated 
as rield experiments. 

r.n^-,!^^ consultants have also come to share some less positive per- 
Zlif u'^^ aggregate of R&D resources, which will be 

amplified in the rest of this discussion. 

1. The numbers of professional and technical staff now available 
for education R&D are markedly inadequate by a number of 
criteria. 

2. The distribution of R&D effort by function and institutional 
base, as well as the coordination or system-quality of the 
whole, leave a great deal to be desired. 

Staffing 

As the U.S. Off ice of Education's former chief R&D planner wrote 
in Science in 1970, "a primary element in educational R&D policy fo- 
cusses on how manpower roles and requirements are defined and where 
those kinds of manpower. can be found."! Though there appear to be lit- 
erally no up-to-date figures on education R&D personnel, extrapolation 
ni/f^H guesses based on professional association memberships 

place the total at about 10,000 people. ^ 

Since this figure no doubt includes many such as university faculty 
whose primary work is other than R&D, there may be only 7,000 or 8 000 

total person-years devoted to education research, development, and'dissem- 
ination. 



1h. Gideonse, "Policy Framework for Educational Research " Science 
4 December 1970, p. 1056. ' ■ ' 

2paisley, The_^tatus_ol^Educat^^^ Research. . . . Chapter ^, Section C. 
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Even if this estimate is low, we are convinced that the absolute 
numbers are too small. Such a human resource base is not nearly large 
enough for sustained inquiry to match the complexity, range, and scope 
of the problems in the schools and colleges of the nation. Several 
different criteria suggest this conclusion to us. 

Tlae inadequate size of the total effort is dramatized by compari- 
sons with research and development in other areas. Striking differences 
quickly are apparent if we make contrasts with the 10,000-person total 
workforce in educational inquiry: 

^ In health, the government's lead agency in health R&D, the 
National Institutes of Health, itself employs over 4,000 
scientific R&D staff, and reaches many more through its 
15,000 contracts and grants to over 1,200 institutions. 

° In energy, a single one of the Atomic Energy Commission 
laboratories typically employs an R&D staff equal to a 
substantial fraction of the entire education R&D group. 

° The National Science Foundation collects data on industrial 
R&D which show that large firms (with more than 10,000 
employees) have 28 R&D scientists and engineers per 1,000 
employees, and smaller firms ('5,000 to 10,000 employees) 
have 19 per 1,000. Public elementary and secondary education 
involves about 2.5 million teaching and administrative 
employees, for a ratio of less than 4 R&D professionals per 
1,000 employees.*' 

Considering the role of R&D within Federal education policy, the 
manpower seems inadequate also. In the last few years, policy-makers 
have described the Federal role in two ways. First, the government 



^ Basic Data Relating to the National Institutes of Health; 1975 (Wash- 
ington, D.C: NIH, February, 1975), p. 45. 

2The Energy Research and Development Administration, Public Information 
Office, gives the estimated employment of scientific and technical staff 
at Argonne National Laboratory at 2,500, and at Brookhaven National Lab- 
oratory, 2,400, 

^National Patterns of R&D Resources; Funds and Manpower in the U.S. 
1953-1974 '(WashingtonV B.C. ; National Science Foundation, 1974), p . 11. 
On teachers. The Condition of Education (Washington, D.C.: National 
Center for Educational Statistics, 1975), p. 173. 
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leads through seed money or the creation of incentives for atten- 
tion to neglected groups and issues through categorical grant pro- 
grams. Second, it will strengthen the foundations of education 
through support of research, development, and dissemination of 
improvements in practice. Whether or not this strategy is a wise 
one — for instance, in contrast to increased general aid to education — 
its success critically depends on the supply of R&D resources. It 
seems clear that a workforce of the size we see has hardly multi- 
plied to the extent necessary to implement the current Federal pol- 
icy. But we are convinced of the long-run importance of inquiry as 
a force for improving education, and if its potential efficacy is 
undermined in the public and professional mind, that will be a major 
loss than even substantially increased general aid programs could 
not compensate for. 

Next, the effort is too small when measured against the decen- 
tralized and fragmented character of the system it is intended to 
influence. It is clear that the production of research, the develop- 
ment of new ideas and materials, by some individuals and groups 
separate from the operating education system is not enough. Publishers, 
state agencies, information systems, journals, and other '^linking 
agents" are, of course, parts of the picture. We agree with NIE that 
dissemination activity will require substantial, increased investment, 
but beyond that, probably every large system must have its own R&D 
capacity no matter what dissemination networks exist. There are 1,600 
school districts with enrollments over 6,000 pupils, and if each were 
to employ a minimal R&D group of five, the nation's R&D workforce would 
need to be doubled, and we will make several recommendations urging 
the Institute to begin to see local and state education agencies as 
R&D performers, not mere recipients or beneficiaries of others' work'\^ 

From these considerations, we do not emerge with a numerical goal 
for R&D personnel towards which to strive. Ideally that should result 
not from ad hoc comparisons such as we have made, but from a program- 
matic analysis of what must be done, and how many people of what sorts 



It has been argued that the workforce in education R&D is at its present 

size as an accurate reflection of the demand from the market, so that 

we are wrong in suggesting an increase contrary to this natural balance. 

We feel the potential of R&D has hardly been given a true test, in view 

of the brief time it has had substantial support, the primitive nature « 

of the dissemination system, and the hitherto weak results from much 

research. It seems fair to call for some further subsidy of R&D under 

stable conditions of production and field dissemination to .at least 

gather further data on the issue of demand. 
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are needed to do it. We hope the enlarged capacity for monitoring 
and analyzing the R&D system which we will recoinmend for NIE could 
begin to provide the data on which such a plan could be built. Such 
data include better figures on the present location and types of 
R&D personnel and analysis of policy issues such as how best to 
strengthen staff for the functions needing to be performed in the 
presently-underserved places such as school districts. 

In the absence of such a plan, we favor the relatively simple 
approach of providing incentives to attract the very best quality 
, . of people into the research and development field, both new entrants 

and those already established in other fields. This approach does 
not set a number of professionals to be reached at a certain date. 
It simply provides a mechanism for the system to expand and then 
throttles down the incentives at some point where intuition and 
collective judgment indicate the system is getting too large. We 
believe this was partly the history of the growth of the natural 
sciences in this country and has worked reasonably well. 

The nature of the incentives is fairly clear, including the 
following elements: 

° Attracting recent graduates into the field because of 
the availability of jobs; this is especially possible 
in the next decade owing to the oversupply of PhD- trained 
individuals. A substantially increased R&D effort in 
education thus can take advantage of a golden opportunity 
to attract some of the best of the current crop of students. 

° Attracting new students by the traditional fellowship 
incentives — though we favor partial self-support by each 
student; despite the current Federal policies against 
fellowships, they are a logical outgrowth of our argument 
about the need for growth of staff. 

° Attracting already-trained persons who are now in other 
fields, but who could make contributions in education R&D. 
This could be done through mid-career education, change- 
of-field grants, or other incentives that support and 
ease the transition. 

Particularly in view of our emphasis in later sections on the need 
for high-quality work, we find the present staff size for education R&D 
^ simply inadequate, and we wanted at the outset of our assessment to call 

this fact to wide attention. 

Our judgment as to the aggregate shortage of R&D professionals does 
not mean we then find all the present people to be of acceptable quality, 
all present institutions worthy of support, or all R&D functions in their 
proper balance and relationship. We do, in fact, have some more refined 
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perceptions as to both the functions and the institutions, and 
about the system's inter-relatedness. 

Functions 

It is difficult to know exactly what functions are presently 
performed in what proportions by the R&D workforce. The 1975 
Databook, (referenced earlier) gives, without explaining the 
method used, an estimate from one sux*vey, which is shown in the 
first column of the table below. We also asked each of the regional 
laboratories and R&D centers to describe each current activity, to 
categorize it as to its major purpose, and to give the number of 
professional staff assigned to each project. A tabulation of those 
reports is shown in the second and third columns below. 

Distribution of the R&D Workforce 
According to Various Sources 

1975 Databook Regional Lab Center 

Type of work Report Survey Survey 



Research 33% 10% 29% 

Development 50% 61% 53% 

Diffusion - Dissemination 17% 3% 6% 



Despite gross definition problems in these figures, the rank 
ordering may be taken for some indication of imbalance. One set of 
criteria of balance are proposed in Sieber's paper in the appendix. 
If only in political terms, the present allocations are undesirable, 
and in terms of testing the true demand for R&D also, the inattention 
to dissemination makes for a weak experiment. Thus, while we hope 
the research sector can grow, we hope the dissemination sector can 
grow even more and that the development staff can be better distributed. 



The totals in the second and third columns do not add to 100% be- 
cause we omitted staff on projects categorized in several domains, 
and some staff were in work outside R&D completely, such as training. 
The figures are obviously weak in another respect, since within a 
"development project" there may be researchers, developers, dissem- 
inators , 
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On the research side, especially in the areas of more basic studies, 
the kind of people who can do good research are relatively rare. 
Therefore, we are not talking about a large increase in this cate- 
gory. But in addition, the nature of development and dissemination 
activities are such that they require group and team work, specializa- 
tion of functions and division of labor, and they must take place in 
many diverse locations. These characteristics set up manpower re- 
quirements of much greater magnitudes than in basic research certainly, 
and even than in most applied research. And especially if systematic 
program analysis and development is increasingly to take place within 
school systems, as we feel it must, and as state and school-district 
leaders tell us they agree, that set of R&D staff will need to be ex- 
panded. 

Institutions 

Turning, then, to the organizations now available as settings 
for education research and development, we find an adequate range 
in existence. Clearly we do need institutional structures beyond 
the universities and service-providers themselves. And we do need 
to *Wke a market" so that good research people can find careers 
other than in the university. Different structures attract people 
of different motivation, also. To cite energy R&D again, those 
national laboratories are able to recruit and hold outstanding 
engineers and scientists, persons who could command much higher 
salaries in industry, because such individuals wished to work 
under the more settled conditions of a government-funded labora- 
tory. On the other hand, there are excellent professional staff 
who do thrive on organizational change in any field. 

Our point is that there arQ different kinds of institutions 
which seem to us necessary to organize the diverse kinds of talent 
education must call upon in R&D. These i include the for-profit 
research, data-handling, testing, and publishing firms; those non- 
profits which also delight in acting in an entrepreneurial fashion; 
the regional laboratories; university-based individuals and groups; 
and the state, intermediate, and local education agencies. These 
seem to make up a balanced institutional complex, or at least we do 
not immediately conceive of a type of R&D setting omitted , and essen- 
tial, except possibly for intramural research at the NIE itself. We 

• will make some comments about the capacities and present utilization, 
and some ways capacities might better be used, for each in turn. 

♦ Independent firms . The entrepreneurs, either for profit or not, 
represent a quick-response capability, and an almost incredible flex- 
ibility of structure and personnel depending upon the market. As 

two of our group reported after a visit to one such place: 

The interesting phenomenon is the difference in expectations 
from what the laboratories and R&D centers have. (The research 
center we visited) is really a business and infused with the 
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ethic of business • It is a supremely confident organi- 
zation which feels that unless the dice are loaded it 
can compete effectively with other comparable organiza- 
tions or labs and centers. They maintain they have the 
ability to attach and detach staff easily as problems 
arise on which government or other parties want help. 
They make extensive use of consultants to get the flexi- 
bility and the expertise they know they need. They appar- 
ently have not only the ability but an enthusiasm for 
managing in a situation of uncertainty. They exist on a 
year-by-year contrac.t-by-contract basis, and want it that 
way. They believe it keeps them on their toes and keeps 
them lean, and as a result they welcome annually facing 
the test of specific contract renewals. Naturally, they 
tend to recruit people who are not devoted to tenure-type 
situations. 

It seems unlikely that any other types of organization will ever 
have the ability to produce research-based analyses with the speed re- 
quired by emerging policy debates, to begin national studies within 
months of the award of funds, or to do any other tasks that are not 
rewarded in the university or where the university time-perspective 
is too long. There are a number of first-rate such entrepreneurs in 
the social science field, and they should not be excluded from the 
government's R&D effort, but should be recognized for their unique 
capacities'. 

Universities . It seems to us that universities will continue to 
include many of the first-rate scientists and scholars who are work- 
ing to extend the basic knowledge upon which education ultimately rests. 
Indeed, because the range of relevant scholarship is so immense, its 
organization for productive work on education is most difficult. Our 
most basic consideration after looking at the capacity of universities 
to contribute to education R&D is that only a few models of such organ- 
ization are evident — the R&D center and the single-investigator, and 

^^e note the interesting case of Educational Testing Service, a non- 
profit agency where an educational product or service (tests) has also 
generated substantial basic and applied R&D in the firm. Until recently, 
perhaps, educational materials publishers spent little on R&D, no doubt 
as their market was not based on factors that could be improved by education 
research. This may be changing, as public systems and their jjuppliers 
feel the effects of pressure for accountability. We also note the 
occasional independent research allowed employees of RAND Corporation. 
Clearly, the location of R&D, and its diverse sponsorship, is a complex 
question which must continually be addressed by NIE data-gathering and 
analysis • 
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that these needlessly limit our ability to use that capacity. 

The need for involvement. in education research of scholars in 
basic disciplines, and the need to move beyond the isolated work of 
individual researchers, led the U.S. Office of Education to establisj 
the university-based research and development canters in the 1960s. 
One of our correspondents, closely familiar with the origins of the 
centers, suggests that whatever the mechanism, there is a continuing 
need for support of basic social science in relation to education: 

From the very start, the program for R&D in education 
took these facts (of little other support for basic social 
science) into account and tried to get good social sci- 
entists to work on education programs... The situation is 
still the same. In fact, the Federal government seems less 
likely than ever to set up a program in support of basic 
social science research, for obvious political reasons. 
And I take it that other funding sources are not likely 
to take up the baton. 

If that is the case, then educational research is still 
in the fix it was ten years ago; it has to smuggle in 
basic research in the social sciences while it pretends 
that its program is designed to concentrate on educa- 
tional matters. Nor does there seem to have been a 
notable increase in such basic research by educational 
reseachers. We cannot, therefore, stop worrying about 
education's external Intellectual bloodstream. 

And in addition to the continuing need for sustained work on 
fundamental problems from a variety of disciplines we ^ould note that 
the organization of the work should include a continuing link to the 
world o? practice. In this way, the advanced students who must increas- 
ingly find jobs outside the university will have a sense of how R&D can 
relate to policy and practice, and in addition, the university work will 
stay aware of the needs and realities of the system. 

However, university researchers are in many ways unsuited to the 
demands of tight timelines and pre-specif led objectives be attained 
which often accompany Federal R&D funds. ^ Problems in education are in 

lAppendlx C lists the present R&D centers, their parent universities, 
and other data on staff and budget. The history of their establishment 
is well summarized in the 1975 Databook , Chapter 4, Section C. 

2 James Coleman's essay on "Policy Research in the Social Sciences" 
(Morrlstown, New Jersey: General Learning Press, 1972) is an excellent 
discussion of how universities should not be expected to perform certain 
kinds of studies Intended to quickly Influence policy. The cancer 
example cited earlier is another Instance of pressure for solutions, seen 
as inappropriate by basic researchers. 
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many cases millennia old, and a university group or any other for that 
matter, can not be expected to find a breakthrough where many have- already 
struggled and made little headway, particularly if we expect the diverse 
approaches of a number of disciplines to be integrated. 

The nine present R&D centers have a range of work under way, 
from heavy concentration on development and marketing of products, 
to more detached work to understand educational activities. They 
vary, also, in the degree of emphasis on a common mission within a 
center. The quality and usefulness of their work varies, as judged 
by educators in the field and scholarly colleagues, and it seems 
fair to say that owing to the diversity- of the work^ it would be 
hard to judge "centers" according to a common criterion such as con- 
tribution to knowledge or help to the field. 

Organizing university talent for large efforts in education 
R&D inevitably confronts issues such as: 

° the balance between autonomy in staffing and ties to 
the depar tmentii* ; 

° independence of action and research vs. closer ties to 
the governnient; 

° term of funding. 

We think that past pressures for rapid development and evidence 
of "impact" have probably forced many centers to neglect basic research and to 
assemble people who could work for immediate results under pressures considera- 
bly different from those in typical academic life. The resulting lack 
of consistent interchange with scholars in the disciplines seems a 
decided loss. Again, the government's need for demonstrable products 
has probably led to closer ties to funding agencies than universities 
typically would wish, and the terms of funding may become shorter, with 
repeated competitions in order to maintain pressure and accountability, 
these developments seem to be barriers to taking full advantage of university 
scholars, and seem almost designed to turn segments of the university into 
independent and lower-status agencies. 

If the R&D center is one model presently in use, the single- 
scholar or small team is the other. Many of these were represented in the 
Field- Initiated Studies competitions held by the NIE two and three years 
ago (in which 82% of the funds went to colleges and universities). We 
agree that diversity of arrangements is necessary, and we do see a 
place for support of researchers who do not wish to participate in large 
organizations. Indeed, the superior researchers in any field who can 
obtain funds easily see no need to have those dollars siphoned off to 
support "overhead," see no reason to join with others of lesser rank, 
and as a result are not often found in "centers." 
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However, our general view of university capacity, and its 
present use, can be tsummarized thus: 

1. The need to support basic work in social science is 
as great today as when the R&D centers were first 
established. 

2. A very wide range of university talent in numerous de- 
partments and professional schools, can be related to 
the national R&D effort in education. 

3. While the university effort in education R&D needs 
for various purposes to be related to the world of 
practice, the work should not be judged by its immedi- 
ate contributions to solving specific problems in 

the field. 

4. Arrangements to capitalize on the talents at universities 
must take a great many forms, not simply centers or iso- 
lated professors, and must take into account the need 
for relative independence and long-tejm support for 
maximum -effective use of that talent. 

5. R&D in education must be organized at universities in such 
a way as to maintain the links with departments in the 
disciplines beyond education which are Its intellectual 
underpinnings . 

The last two points are most important to us. We feel it is part- 
icularly essential to imagine ways of relating to universities that 
draw on the best people there, rather than relying exclusively on a separate 
entity which establishes its own staff on soft funds, potentially unre- 
lated to the academic and intellectual life of the rest of the university. 
The arrangements used must be apt for engaging the present scholars, 
where they have contributions to make, not establishing a second-class 
set of citizens, impermanent and isolated. 



Sam Sieber of our group has argued in his book Reforming the University ; 
The Role of the Social Research Center (New York: Praeger Publishers, 
1972) that the "integrative functions" of university research centers 
are severely hampered if such agencies are not able to operate as secure 
and stable entities. He sees this integration taking place as the cen- 
ters span boundaries between usually distinct areas: university and 
society; research and service; student and teacher (through research 
apprenticeships); education and basic disciplines; intellectual work 
and management. 
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In addition to persons in regular departments and the schools 
of education, there are other resources in universities not much tapped 
at present. These include:, 

o Schools of business, which have a growing interest in 
public management; 

o Policy-science departments or institutes, where people 
are coming together around general issues in the design, 
implementation, and evaluation of public programs; 

o Schools of public administration, where focus is shift- 
ing towards policy-making, as well as its execution. 

We do not have a clear formula to suggest that will in all cases 
hook the right parts of universities together for various education R&D 
purposes. We do have a sense that more options should be explored, as 
the present capacity is under-used and its procurement bound essentially 
by two limited concepts — the center and .the individual or small project. 
We have in mind a least two other modes. 

1. An up-do-date survey should be done of university faculty 
and the departments, schools, and institutions in which they 
work to determine the-* - potential for an interest in educational 
R&D. Based on a determination of high potential, the precise 
form of the organization could well be left to the university 

or a major segment of it, with considerable discretion left to 
the grantee to put together a combination of people in whatever 
form and manner is appropriate to it. If this turns out to be 
through an established R&D center, so be it. If the mechanism 
is an internal university foundation, so be it. 

2. When the government defines certain work needing to be done, 
and finds scattered university resources available that may 
not naturally come together in a joint effort, procurement 
rules must be interpreted to allow the government to bring 
the parties together, broker agreements, and emerge with a 
total resource greater than the parts alone. 

We have been told several times of the success of the National Science 
Foundation in drawing world-renowned university figures in Boston and 
Cambridge into science curriculum development. The method was a loose, 
time-limited entity called Educational Services Incorporated (ESI) , which 
had a few central services, but chiefly organized the part-time efforts 
of many others from the university community. While we feel this example 
is in many ways unique, it does underscore a general point, that education 
R&D in the present era need not hold to one or two ways of organizing 
university resources. We find special merit in the argument drawn from 
this example, that we need not be bound to permanent institutions, 
in universitites or anywhere, that live on after their missions are 
completed. 
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When we return to specific NIE policies, we will discuss further 
our notions of reducing the present number of centers, which would 
give room for creating a variety of new arrangements that suit the NIE 
program areas and are better adapted to the special circumstances of 
universities. 

Laboratories , The third set of institutions whose capacity we 
must address is the regional educational laboratories, established by 
the U.S. Off ice of Education upon passage of new legis^^1^ive authority 
in a 1965 amendment to- the Cooperative Research Act. As the Background 
Report on the "Labs and Centers" -^ outlines, twenty were created ori- 
ginally, dispersed throughout the country as the name implies. As 
appropriations failed to increase, nine were closed by USOE in 1970, and four more 
have ceased to receive Federal funds since then. Even with these reduc- 
tions from the original numbers, by the time of our survey of the re- 
maining seven laboratories in April 1975, they employed almost 700 
full-time equivalent professional staff, or close to ten percent of 
our estimate for the nation's entire R&D workforce. Further, their 
annual budgets in FY 1975 totalled over $20 million, $18 million of 

n 

which came from NIE.^ 

Our general feeling, after considerable listening to leaders of 
the laboratories both together and individually, examining lengthy 
questionnaire returns, talking with past and present Federal govern- 
ment staff, and having the benefit of the written views of several 
dozen observers of education R&D, is that the concept was distinctive 
and important when first broached, and remains sound. However, we 
are wide of the mark in many ways in the present implementation of it. 

By "the concept," we mean at the most basic level, the idea of 
establishing long-term R&D institutions distinct from both the univer- 
sity and the operating education system, and not in the market to do 
odd jobs of research or service, but to carry on substantial work on 
complex problems. The task force which originally framed the idea 
had in mind the creation of a small number of high-quality national 
laboratories comparable to those of the Atomic Energy Commission, and 
perhaps with other features similar to some of the clinical facilities 
of the National Institutes of Health. 



Prepared by NIE staff for the National Council on Educational Research 
for its March 5, 1975 meeting. 

Appendix C gives further data on laboratory staff and budgets. 
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^ How has the concept weathered a decade of efforts to implement 

it?^ One former official of the U.S. Off ice of Education who was close- 
ly involved during the creation of the laboratory legislation offers 
thi| further personal recollection of what the laboratories were to 
be: 



The laboratories would capitalize on (the potential of 
great advances in basic knowledge) by devoting atten- 
tion to basic research but would devote prime attention 
to "development and dissemination of educational inno- 
vations." Tne laboratories would have strong links to 
state departments of education, to school systems (par- 
ticularly for teacher training and field testing), to 
universities and industry. Every laboratory would have 
one or more experimental schools "more or less under its 
own jurisdiction." Together, the laboratories would 
form a' nationwide network to test the feasibility of 
new methods. They would also establish effective chan- 
nels of comiuuuication among themselves, collaborating 
easily and continuously. Teacher training was to be . 
an integral and major part of the lab program, with 
new models for the education of teachers emerging in 
the process. 

Despite these specific early hopes, the laboratories in fact came 
to enjoy a relationship with the U.S. Off ice of Education in which they 
proposed work they wished to carry out, and the Office negotiated and 
agreed. The Federal R&D officials lacked sufficient time or mandate to 
develop long-range priorities against which to judge laboratory work 
plans, and the initiative thus rested effectively with the labs them- 
selves, in competition only with each other for shares of a specific 
segment of the USOE budget. In this -ay, laborator-ips vcre able al- 
most CO set their own agenda, and to come up with their own best sense 
of what functions Lo perfoxrm, for whose benefits, with what tools, 
Each institution worked under a single contract. 

In 1972, the USOE decided that this relationship should be 
changed so that the government would henceforth be purchasing specific 
programs of work, under separate contracts for each' program. A contract 
price and term would be negotiated at the outset, with products and a 
schedule for delivering th«m. The NIE inherited this incipient policy, 
and concluded the latter part of a review process begun at USOE tc 
decide what specific work should be supported and for how long. 

■^Samuel Halperin, "Title IV of the Elementary and Secondary Education 
Act of 1965: Executive Aspirations and Legislative History." (Pre- 
pared for presentation at the 1971 annual meeting of the American 
Educational Research Association*) 
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Thus, the laboratories were seen -at first as a unique institutional 
capacity, able to plan and set their own direction subject only to constraints 
of budget and occasional internal and external reviews. As funds stabilized, 
as judgments had to be made about terminating Federal support at some 
laboratories, and as specific programs of work have become the focus, 
questions about the cost-effectiveness of this form of R&D and the impact 
of its products hnve been raised more and more. In this second era of 
relationships, the laboratories seem to have been viewed as organizations 
like any other, which had agreed to do a job, and should be judged on the 
work being done. In a sense, the government could argue that its R&D 
Toanagement approach has matured, from a time when simply establishing 
some new institutions was in itself the major challenge, requiring 
considerable discretion left to the field to carry out the novel kinds 
of work everyone had in mind, to the recent years when managers felt 
more confident in the assumptions that the system could take care of it- 
self, that adequate institutional capacity existed, and that the focus 
for federal funds could shift to major R&D priorities determined after 
wide consultation with both practitioners and researchers. 

As a legacy of the early years of laissez-faire , the seven current 
laboratories differ enormously from each other, and the specific work 
they were contracted to do after the 1972 review reflects this diversity, 
as it was largely based on what had gone before. So we find laboratories 
at present varied in their sense of what schools and students need, 
in their internal governance and policy-making, in their ties to their 
religion, in the degree of emphasis on service to the local schools 
nearby, in the balance of functions performed such as research, development, 
dissemination or evaluation, and in the degree of programmatic coherence 
and mission emphasis. Also, they differ in their current degree of 
dependence on the NIE. 

Thus the laboratories are a patchwork of capacities and interests, 
and though we could hardly evaluate each laboratory against a set of 
criteria, our impressions (and those of our correspondents) were of 
very diverse quality with respect to any specific capacity such as 
product development or evaluation. In this connection, we recorded from 
our discussions in the field some serious issues and criticisms concerning 
the labs' present capacity. Among these were: 

o Major resources devoted to curriculum designs and materials 
of instruction, without adequate study of the schools' need 
for them or their appropriateness to real problems. 

o Little sophistication in marketing and sales, and unfortunate 
relations with commercial publishers, one of whom described 
as follows relationships with a laboratory: "we were summoned 
like errand boys, assumed to have no educational views, 
editorial abilities or traditions, but lots of money — and 
told to peddle these brilliant programs. 
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This style has probably changed in recent years, but the 
memory lingers on. Unless a better relationship is est- 
ablished in which the publishers are asked to the dinner 
as well as the show, I doubt if the best "dissemination*' 
instrument available will be made effective." 

Actual products themselves often not demonstrably better 
than the range of commercially-developed materials, and 
frequently more expensive. 

With certain outstanding exceptions, laboratories per- 
ceived by important practitioner groups (superintendents, 
teachers) as distant, unhelpful, essentially similar to 
other consulting or contract-seeking entrepreneurs. 

Too strongly wedded to conceptions of educational change 
through hardware and software; not adequately interested 
in system-change and change from outside the establishment 
(even granting the limits of R&D in these areas). 

Without close ties to universities, seen as not equipped 
to exert academic forms of quality control, nor closely 
regulated by boards or reviewers, and the result is a very 
wide range of quality of staffs, coherence of missions, 
tautncss of management, and utility of products. 

Some of the most devastating commencs about the laboratories 
came to us from teacher and administrator groups, which is particul- 
arly troubling in view of the substantial funds already spent in 
developing products aimed at helping these two groups. 

Nevertheless, despite the criticisms we have weighed, for us the 
only question about the basic concept of the laboraturies Is how to 
make it work well, not whether the laboratories should exist. The 
need for established, long-term, R&D institutions still impresses us. 
And we believe that a good many of the complaints about the quality 
or orientation of the remaining labs must be laid directly at the door 
of the Federal government. The Federal government created the institu- 
tions to meet certain needs, and if the needs are not being met as well 
as they mi^..^., it seems plausible to us to examine the Federal govern- 
ment's leadership and management, rather than to reject the concept' of 
the laboratory structure. 

However, saying we believe the need still exists, and that the 
original concept of distinct organizations to meet it is sound, we 
must hasten to add that at present the concept is out of control 
and being implemented in unsatisfactory ways, and that there are still 
too many institutions (given the shortage of quality R&D personnel) 
to insure the uniformly high quality of work originally hoped for. 
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Furthermore, because the federal government's education R&D expenditures 
have not risen as many hoped, and because it is important to invest in a 
variety of approaches to education R&D and to consider the proper pro- 
portion of funds that should go to any particular set of performers, 
we come to the conclusion that the laboratory program must be limited 
to allow reasonable support for the activities of other important agencies. 

In thus reconsidering the concept of the laboratory and how to . 
move deliberately from the present situation to the future, we have 
in mind certain principles which we think should charactelfize tjie 
capacity of the "new" national laboratories which we expect to emerge 
in the next few years as one key part of the R&D system. 

First, there should be a small number of very high quality in- 
stitutions, perhaps no more than a half dozen. Though vfe later pre- 
sent some budget considerations, these are not controlling. We think 
there are only so many top-flight staff available for such places, 
which sets a limit on the number that should be supported, and we also 
believe that other institutional arrangements deserve attention beyond 
the one we are considering here. 

Second, each institution should center on a mission, closely 
related to a priority of the major sponsoring agency, the National 
Institute of Education. The work of the organization would be to 
take ideas from their inception through their development, refinement, 
and testing where appropriate, to dissemination. The substance to be 
worked on could range from classroom instruction to systems of school 
finance. 

Third, the organizations' funding must meet several standards. 
It must be stable (three to five years), it must chiefly come from one 
source (the NIE) , it must be clearly tied to specific work agreed upon 
in advance, and it must be of a magnitude of $3 to $4 million per year 
at least. We believe that quality work on complex problems requires a 
clear sense of mission, relative freedom from the distraction of 
searching for funds, and the assembling of a sizable team of people 
that can work together over time. 

Fourth, the organization must be protected from demands to give 
undue services to local and state agencies unrelated to the major R&D 
mission. Any tie, such as to a state legislative committee or local 
school board, should be in the service of working out an idea °^ J^^<^^ 
applicability, as a way of improving the national R&D product. Thus 
individual organizations might or might not be working with agencies 
near them. Testing, for instance, could be going on in a distant state. 
We believe such ties to the field at all stages of inquiry are essential 
for the laboratories' purposes, but we are cautioning against seeing 
a small number of national laboratories as places where practitioners 
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might go and expect advice on education problems generally. 

Fifth, it should be clear that the unuiual guarantee of contin- 
ued funding will bring with it a need for an unusual degree of monit- 
oring and review of the work. Much more than at present, an organiza- 
tion which wishes the kind of stability we have outlined here must 
expect close review of policy and management by the sponsoring Federal 
agency, NIE. The special relationship of insulation from repeated 
competitions for short term funding makes other forms of quality control on the 
substantive work especially important. The relationship of policy boards to 
their institutions may need reexamination and change as the Federal role 
and mission become more significant in the renewed laboratories. 

Sixth, the pursuit of other .funds by the organization should be 
subject to review and perhaps limitation, in order to insure the 
focus on the major goals of the chief sponsoring agency, NIE. Addi- 
tional funds which support activities closely related to the main 
mission would be welcome, but active search for other types of funds 
would raise questions about the commitment to the major mission. ^ 

Seventh, the redesigned laboratories would no longer be desig- 
nated "regional," though we do feel they should be located in dif- 
ferent parts of the country. So long as they expect substantial 
Federal support in an era of very tight Federal R&D dollars, they 
must be working on parts of the national R&D agenda, as set through 
a national process. The institutions would, of course, contribute 
to that planning process, through their knowledge of local issues, 
but they would not be expected to independently address local problems 
through R&D. 

Eighth, lastly, we are concerned that as such strong and unique 
institutions mature further, they maintain a prime commitment to effect- 
ing change and improvement in schools. Their closeness to the govern- 
ment, and their somewhat detached R&D work may reduce the vividness of 
the daily life of students and teachers in classrooms. We heard too 
much criticism, on this score concerning the present laboratories and 
centers, and we note the unfulfilled aspects of Halperin's list of 
original hopes — close ties to teacher training, maintenance of experi- 
mental schools, and constant attention to have R&D impact pressing 
needs of students and teachers. No amount of sophistication in the 
R&D work can compensate for irrelevance to the world of educational 

H/e do not rule out such funds, and we are aware of the Commission on 
Government Procurement view that institutions such as we are proposing 
should not receive all funds from a single agency. ( Report of the 
Commission, Volume 2, p. 18; Washington, D.C.: 1972). Nevertheless, 
we feel strongly about the need for consistency and goal orientation 
of funds, and outside review may be needed to maintain that. 
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practice, and the renewed laboratories should not forget this. Even 
such small details as the lavishness of facilities and the size of 
salaries may set a tone of separateness that can be harmful. 

The difference should be clear between these aspects of our con- 
cept of the national laboratories of the future, and either the earlier 
U.S. Office of Education notion of supporting independent institutions 
which set their own agenda, or the current NIE concept of purchasing 
discrete products from an tmdif f erentiated set of institutions. 

Because of the relatively small capacity in the education R&D 
system generally and the continuing need for substantial long-term 
attack on educational issues by R&D professionals, we see no alternative 
to the deliberate use — and reshaping where necessary, as we have argued — 
of capacity built up over almost a decade. What should happen in the 
laboratories is that they must be given a sense of direction. They 
should be staffed, organized, and operated in a way that meets the pro- 
gram priorities of the major funding source. We do not mean to imply 
that the national laboratory is the only mode of R&D effort, and we 
turn in a moment to discussing the need in the next few years of strength- 
ening the capacity of other persons and institutions to do R&D. 

The notion we have of the new or revised institution is rather 
clear to xjs, and we do not wish to be misunderstood on several key 
points: 

° We do not expect many current university centers to desire 
the expanded funds and special relationship to NIE that we 
propose, nor do we think that universities could actively 
involve only their best faculty in programs of R&D at the 
$3 - $4 million level. Thus, while we encourage a review 
of current centers on the chance that some might seek such 
a relationship, we do not think a negative decision on that 
score should rule out other kinds of ties and support for 
university work and training, such as we have spelled out 
above. Many other Federal agencies maintain centers of 
research at universities on a scale larger than the indi- 
vidual project, without creating the relationship we have 
spelled out for great national laboratories. 

° The renewed laboratories, whether drawn from present centers 
or laboratories, should indeed span all functions from re- 
search to contact with ultimate users; such a vertical in- 
tegration is one reason for their expanded funding. We do not 
mean, by using the term "laboratory" which has connoted 
an institution chiefly expected to concentrate on product 
development, that this function should predominate in the 
renewed laboratories. (See our couflaents later on the whole 
concept of development and how it should be expanded.) 
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Nor do we expect the national laboratories to isolate 
themselves from contact with, and assistance to, the 
education system. Good R&D requires a constant inter- 
play of action and theory, mediated by a process of 
design to trafis3,ate theory into concrete systems, be- 
haviors, and prdducts. Effective dissemination, adapt- 
ation, and utilization of R&D outcomes will of course 
require laboratories to be in touch with the ultimate' 
users, as well as with linking agents who are helping 
users find relevant R&D. We do not think, however, 
that good R&D flourishes in an environment where for example 
sheer survival of a laboratory requires taking on a 
dozen small field evaluation projects for local schools 
which are of little interest or stimulation to either 
party, or the performance of training in a field far 
removed from the laboratory's mission. 

We do not imagine that all current laboratories or R&D centers are 
doing the best job that can be conceived; but we feel strongly that one - 
builds on what exists; and one moves to strengthen that base towards the 
goals of greater quality of work, relvance to the missions of the sponsoring, 
agency, and aid to the operating system. Casting the present resource loose 
into an increasingly stormy sea of competition for small or short term erants 
seems far from a deliberate use of the present set of Institutions, which 
represent such a cost in trial-and-error learninc already. We will return 
to the specific issues of how these capacities at the laboratories have 
been dealt with by NIE, and how our eight-point concept of .laboratories ^ust 
described could become the basis for new NIE policies, in the last two chapters. 

Local School Systems and State Education Agencies . It is a 
common observation that the operating educational agencies invest ~ 
little in research and development, at least as that activity is 
narrowly defined. The 1975 Databook offers little information on 
the subject, quoting some tenuous guesses. An earlier paper by NIE 
titled Building Capacity for Renewal and Reform described a tele- 
phone survey of large-city districts which turned up only a small 
fraction of one per cent of any budget devoted to such activity. 

YeL there is more than that happening, and we feel NIE must 
somehow build upon the present effort. For instance: 

Many states have established planning and evaluation 
offices using funds from Title V of the Elementary and 
Secondary Education Act (ESEA) . 

State and district accountability schemes involving the 
analysis of tests and other data are multiplying. 

States conduct evaluations of Federal programs such as 
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Titles I and III of ESEA. 

° Some city districts have substantial research and plan- 
ning offices, with sophisticated data systems, curric- 
ulum development projects, and internal dissemination 
efforts. 

° city and state agencies are invblved with a variety of 
outside helping organizations for research, surveys, 
\ staff training, organizational development, and evaluation. 

We are concerned about two linked aspects of this situation. 
Since little is known about the R&D activity in such agencies, little 
thought is given to strengthening their ability to perform it. As a 
result, there is continued the traditional image of the practitioner 
and the operating system generally as the recipient of disseminated 
information, produced elsewhere for their benefit. While this is be- 
ing modified slowly, as developers recognize the need for local adap- 
tation of even the finest tested products, there still needs to be 
a fundamental recognition of the latent capacity for systematic think- 
ing that exists in many syst3ems. 

Given the meager resources for educational iiiquiry and program 
development, it would be an unfortunate waste to write off the present 
scattered and sometimes unsystematic efforts of state and local agencies. 
Of course there are a dozen barriers, such as lack of skill or bias to- 
wards the status quo, that might interfere with the ability of public 
school agencies to look closely at themselves and take remedial action — 
negative responses in some places to the Coleman survey, the National 
Assessment Program, the voucher experiment, and other potentially re-- 
vealing or provocative R&D efforts make that clear. 

"^As we noted in a parallel argument under the discussion of staffing 
for R&D above, it can be said that a present condition of little in- 
vestment by school systems in R&D, or even in program evaluation as a 
first step towards systematic inquiry, accurately reflects the incen- 
tives and rewards for such work. While we agree that there never will 
be incentive for a single system to do basic studies in children's learn 

I ing, we believe that other kinds of localized program development and 

assessment activities are useful and proper, and that many systems are 
working in this area. The Federal role is to support good examples, 

, study the conditions of their success, and subsidize new versions to 

refine the analysis. As with the argument under personnel, we do not 
think the concept of school, district, and state-level inquiry has been 
adequately tested. For a pessimistic argument, but no data, about the 
potential of stich a test, see John Pincus, Incentives for Innovation in 
the Public Schools (Santa Monica, California: RAND Corporation, 1973. 
Publication P-4946.) 
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Nevertheless, we will return in our recommendations to some 
concrete ways in which Federal R&D policy at NIE could begin to engage 
the xncxpxent capacity in local and state education agencies, both 
tractr"'%f to sensitize other grantees a^d con- 
tractors to the need for close work with them. Particularly if we 
are encouraging a more specialized (less service-oriented) set of 
institutional capacities in universities and national laboratories 

Z^ZTllllTrT^: -^^ ^"^"^''^ ^° it-lf "hat others' 

may no longer be doing for it. Finally, on this topic, we war,t to 
recognize that some local school districts have built up admirable 
nJIn^^r^^'^ '° f"^^ '^^^'^ °^ problems and design solutions. NIE 
lltt^" ^^T^ these and to encourage such 

activities wherever they can take root. 



System Qualities 

As a final aspect of the present resources for education R&D, 
we want to comment on the degree of "system" that seems to exist. 
We have been impressed, even though we have far too little data, to 
find so little interrelation of parts. There is an aggregate of in- 
dividuals, institutions, and other resources that combine and relate 
m a variety of ways to a variety of ends. Such an observation 
strengthens our view that a single Federal agency, with limited 
funds, must not be expected to have massive impact on such a "system" 
in the short run. But it also reinforces our conviction that con- 
siderably greater effort must be put into mapping the pieces of the 
universe, understanding the diverse structures and their internal 
strengths, so that deliberate policy toward' each part can be attempted, 
and so that potential connections and interrelations can be encouraged. 

In this connection we note papers on the R&D system by Ronald 
Corwin, by Egon Cuba and David Clark, and a thoughtful letter sent 
us by Ward Mason of the NIE staff. ^ Corwin describes education re- 
search as a loosely-knit social system" made up of "disparate and 
nebulous communities" of which many are now in an "anomic state." 
Noting that "science tends to be rather disorganized," Corwin re- 
counts the history of research management and concludes that attempts 
to impose more rigid forms of organization have not clearly improved 
the quality of the product. 

Cuba and Clark join Corwin in both describing the fragmented 
system and analyzing how poorly past policy reflects the "reality." 
They suggest an alternative framework to the notion they feel has 

■'■Ronald G. Corwin, "Beyond Bureaucracy in Educational Research 
Management," AERA Division G newsletter, "The Generator," Winter 
1975; Egon G. Guba and David L. Clark, "The Conf igurational Per- 
spective: A View of Educational Knowledge Production and Utiliza- 
tion (Washington, D.C.: Council for Educational Development and 
Research, 1974). 
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reigned unexamined in recent years, of a mechanistic system of 
specialized institutions in research, development, dissemination, 
in orderly sequence- They urge that future policy planning take 
into account a number of currently overlooked aspects of the real- 
ity of the R&D system: 

° Knowledge-production and utilization is a secondary . 
goal for most institutions in the system, and as a 
result the more basic missions of each place will not 
be dropped or altered in favor of more systematized 
R&D or other externally-imposed goals. 

° No institution likes to be placed at the receiving 
end of a system—or at the bottom of a prestige hier- 
archy—as local schools are in the traditional system 
model. 

° Institutions typically refuse to remain specialized, 
and will expand to take on related functions, thus 
blurring any permanent distinctions intended by the 
central planners. 

Mason adds that wide diversity of approach i^ inevitably to be 
found in a field like education which lacks accepted "paradigms 
or models of what is important to study and how to do it. 

These points seems excellent advances toward a description of the 
aggregate of people and places doing education R&D, though we disagree 
with these^^authors in drawing normative policy conclusions from the 
analyses. (For instance, we do not concur that the proper NIE res- 
ponse to fragmentation of the R&D community is to delegate much of its 
own decision-making to that community.) The important point to be 
made here is that the system— no matter how one comes eventually to 
characterize it, as community, organic body, machine, or whatever— 
is not well-known at present. The sketchy data ^^^^^^^^^ vlvlous 
support a number of interpretations. Informed choice among ^be varxo^^ 
instruments of policy, from laissez-faire , encouragement of colleague- 
ship and networking, to central planning and direction, across the 
various domains and types of R&D work, must be based on far better 
understanding than exists* 

For additional discussion of the nature of the R&D system, and 
the needs it should be designed to serve, see the paper prepared by 
Sam Sieber of our group, which appears in Appendix A. 

Having said some things about the resources as we see them now 
and how they might develop in the future, we must come to consideration 
of the role to be played by the National Institute of Education. In 
the next two sections we take up the present context of forces acting 
on NIE, and the policy directions being selected. 
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IV 



THE CONTEXT FOR 
POLICY-MAKING AT 
NIE 



General expectations for an activity like R&D, and a set of 
available resources such as we reviewed in the previous section, 
are two ingredients for R&D policy. But at a specific time, policy 
choices must be made within a confining context of forces at that 
moment. Our review of funding policies at the National Institute of 
Education was called for at a particular time, because of particular 
pressures, and we felt the need to inform ourselves about the present 
environment of NIE. Thus in the section which follows we touch on 
organized groups. Congress, NIE staff, the National Council on Edu- 
cation Research, other parts of the executive branch of the governnent, 
and finally a force from the past— the weight of inherited commitments 
and staff that the new agency had to carry from the outset. 

Alternatively, the discussion could be approached through certain 
issues or questions to which the Institute must have practical answers 
and which are the focus of debate and concern by groups and individuals, 
within and outside the agency. These include: What should be the 
goals of NIE? Who should participate in decisions on what to do? At 
what points and with how much relative influence? Who should perform 
the Institute's work? What methods are best for doing that work, ac- 
cording to what criteria? Who should benefit from the work? By what 
standard should "benefit" be discerned? How should staff be arranged 
Internally to manage the goals selected? 

These matters have been in dispute almost from the moment the ink 
was dry on the authorizing legislation. For some time the basic policy- 
making machinery of the National Council was not ready, yet the staff 
needed to make decisions. A self-appointed private watch-dog group 
threatened lawsuits over allegedly illegal policy-making procedures 
within months, and the new Director had to try to defend himself and 
the administration in early oversight hearings before the House author- 
izing committee. As various parts of the NIE program have emerged in 
later months, a variety of others have begun a steady volume of com- 
plaint that they had not been consulted and should have been, or that 
the proposed directions were hopeless in any case. Staff, recruited 
from academia or other Federal agencies, may have hoped to revive in 
the fresh and youthful agency the spirit of educational change— now 
flagging— that had characterized earlier administrations. They were 
no doubt impatient with lengthy and participatory planning processes. 
The fluidity of the early years had the result that hardly an educa- 
tional journal or professional newsletter has not had at least a few 
"look-at-the-mess-at-NIE" articles. A vicious cycle of criticism, 
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budget cuts, demoralized and lackluster performance, and still more 
criticism has only recently abated. 

We review some of the actors and forces in the drama, as we 
see them today. 

Organized Groups and Associations 

\ .u^^^ relearning the lessons of the years of work leading up 
to the legislative victory for basic Federal aid to education in 
in 1965~that a great many groups and factions must be considered, far 
beyond an act or agency's immediate beneficiaries, simply because in 
education politics so many people have their hands on the brakes or 
feel that their hand ohould be on the throttle in preference to others. 

Thus NIE has to deal with teachers and their organizations, 
anxious to be consulted at every step and alert to watch out for 
radical experiments that might disrupt important parts of their pro- 
fessional world. It must be aware of higher education groups who 
monitor the balance between support of university scholarship and 
contracts to non-profits and who wonder why little attention is paid 
to their own problems of post-secondary schooling. It must at the 
same time coi;tendwith organized non-profits who, in turn, remind 
NIE of the poor image that practitioners have of university research 
and of their own responsiveness to government demands for new kinds of 
evaluation and product development that universities never got into. 
Chief state school officers— at least their leadership and Washington 
officials— lost faith in the NIE earjy on, as a growing role (and 
growing dollars) promised by the USOE in product dissemination seemed 
nipped in the bud when the work of the National Center for Educational 
Connnunication was transferred to NIE and the states were not immediate- 
ly brought in as they had been at USOE. The chiefs' Council set up a 
special committee to keep an eye on NIE, and played the continuing 
role of skeptic and critic of NIE's budget in the annual deliberations 
of the Full Funding Committee (setting the lobbying posture of all 
major educational organizations on appropriations) based on their view 
of NIE s lack of responsiveness to states. 

For their part, scholarly associations in sociology, psychology, 
and other disciplines whose hopes were raised by a first-year discipline- 
oriented and field-initiated research-grant competition at NIE, grew 
concerned when the following year's competition was much more directed, 
and when no further field- initiated competitions were held. As avail- 
able funds went to existing development activities or directed requests- 
for-proposals, university scholars no doubt saw these as excessively 
practical activities. The educational researchers themselves have been 
in a longstanding quandary about how much of an activist role to play 
in Washington, thus presenting no united front to the NIE. 

Thus the various organized groups differ in their notions of what 
proper to be studied, how best to help improve education in the first 
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place, who the proper performers are, and in short — about almost 
every key aspect of an Institute's life* A rising appropriation 
might have permitted successive approximations to each group's 
expectations, but that has not been the reality, and zo the extent 
the debate is over money, the groups now must fight over a static 
budget, where someone's increase is someone else's loss. 

The strongest organized voice in the environment at present 
is the consortium consisting of most regional educational labora- 
tories and research and development centers (CEDaR, for the Council 
on Educational Development and Research), The lobbying position of 
these institutions is that their unique historical status as Federally- 
initiated R&D performers, and their present usefulness to the world 
of education, argue for a major continuing NIE commitment to them. In 
anticipation of continuing decreases in their share of NIE funds, the 
group has lobbied vigorously in Congress and with the other associations 
for explicit legislative direction to be given to NIE to continue their 
work. Pressure on the House of Representatives resulted in a direct 
earmark of NIE's fiscal 1976 appropriation for the laboratories and 
centers, but even greater pressure on the Senate resulted in less re- 
strictive language. And, in the service of their undenied self-interest, 
this group of institutions has been almost single-handedly telling the 
story of education R&D on Capitol Hill, 

It is fair to say that because of the important political ad- 
vantage of the laboratories' and centers' geographical diversity, the 
present size of their budgets (which makes them significant installations 
in any Congressional district), and' the sheer persistence of their cam- 
paign on all fronts, the CEDaR group members have been the single most 
important continuing pressure on policy at NIE, They work with Congress, 
with the Council, with staff, and have written and called us a number 
of times outside our scheduled meetings. As we understand IL, and to 
simplify, laboratories typically claim that their products have been 
carefully tested and enjoy wide appeal. Research centers argue that 
critical masses of diverse researchers have been brought together for 
mission-oriented research over periods of time that would never have 
been possible under individual grants and contracts—as was the intent. 

Both argue that the Institute has a Congressional mandate to build 
an R&D system, and that the USOE began that task with the creation of 
the labs and centers, with promises of continuing support. It follows, 
they would say^ that a clear way for NIE to carry out the mandate is 
by continuing the existing units as keystones of a system. Labs add 
the political note that continued funding of their development efforts 
will in turn rebound to the Institute's political benefit as consumers 
see more and more products as fruits of the R&D dollars. Thus as educa- 
tion needs products and labs make products (better than others), that 
activity should continue at a high level of support. Needless to say, 
these propositions are not supported in all quarters. 



40 



-36- 



ERIC 



The Congress ' 

NIE was created in the House, added to a bill (the Education Amend-- 
ments of 1972) fundamentally concerned with creating the new structure 
of Federal aid to students in higher education— a bill most contro- 
versial for its anti-busing riders. Despite the fact that the need 
for a national institute of education had been advocated by a govern- 
mental task force and numerous individuals, with these other issues 
claiming significant Congressional attention, the NIE emerged from 
the legislative process with few backers and friends, and without much 
direction beyond the enormous hopes of the statutory language and 
accompanying reports. This substantive ambiguity, and the huge auth- 
orization ($550 million over three years) probably fueled the hopes, 
which may have to be damped somewhat, for something for everyone and 
for rapid improvements in schooling directly attributable to NIE 
efforts. Sophisticated statements from a Daniel Patrick Moynihan 
about the need for decades of effort in basic studies before any 
payoff were buried in the testimony, not vividly kept before the 
members at passage or since. 

So without widespread Congressional commitment or understanding 
of a mission, but with potentially enormous expectations and a 
shrinking base of Federal education dollars to bid for, the NIE has 
run the Congressional appropriations gauntlet, and that story is by 
now legend. The budget has been in trouble ever since some unhappy 
early encounters between NIE*s first Director and key Members of Con- 
gress. From a first-year "honeymoon" spending level of $142.6 million, 
the President's budget request for NIE shot up to $188 million in 
1974, but a bare $75.7 million was granted. The amount dropped even 
further, to $70 million in the third year, 1975, against a request 
of $134 million. Even then, the result was a cliff hanger, as the 
Senate voted zero dollars" and the House reduced its own Appropria- 
tions Committee recommendation from $100 to $80 million after passion- 
ate floor speeches against research by Edith Green and H.R. Gross. 
As confidence in the agency diminished, report language has begun to 
give direction to NIE, culminating in the present deliberations over 
the 1976 budget, where close to one half the budget was earmarked by 
the House, as the price of granting the modest $80 million budget re- 
quested. 

Though there have been no oversight reviews by the authorizing 
committees since the early probing about the National Council appoint- 
ment delays, the appropriations committees have commented on the sub- 
stance of the program while cutting budget requests. Questions at 
the hearings and the committee reports indicate a general skepticism 
about the work under way, its practicality and utility to "those on 
the firing line," and a strong concern for more dissemination of what 
is already known. Criticism has been voiced, more in private, about 
the incomprehensibility of witnesses, documents, and programs them- 
selves, though in recent months that has been reversed and the Insti- 
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tute has for the first time been praised for its increased "goal 
orientation" and an understandable program structure. This history 
of criticism and pressure on the budget by Congress had severe impact 
on the agency and has led to reducing the political risks taken at 
every possible point. 

The relation of the Institute to Congress includes several other 
elements that may be worth adding for a complete picture of forces 
exerted: 

Some individual Members give strong hints about their 
interest in seeing certain projects continued, independ- 
ent of their potential for research. 

Some Members and staff on Capitol Hill simply wish that 
NIE would just clean up its public act, so that clamorous 
lobbyists would go away. Several on Capitol Hill have told 
us, "Why can't NIE just arrange some kind of truce among 
these different groups? We're tired of getting calls and 
mail on this; NIE has to solve its own problems." 

The Institute has been directed to carry out certain 
studies by Act of Congress. These include a study of 
crime in the schools and a more extensive review of com- 
pensatory education. 

Some of those most closely connected to the original design 
still hope that NIE can live up to its mission for superior 
quality research, done by the most able people available, 
drawn from the widest range of fields such as neurophysi- 
ology and nutrition. 

Finally, we understand that there is in the background some 
continuing interest, in the Senate especially, in another 
reorganization of the whole education segment of the Depart- 
ment of HEW, including the Assistant ^Secretary, National 
Center for Educational Statistics, Office of Education, and 
the rest. Public witnesses at recent NIE reauthorization 
hearings made suggestions of this sort also. 

Congressmen typically hear from their chief state school officers, 
local superintendents, and teachers' groups, all more or less tied to 
present educational practice and immersed in today's educational prob- 
lem — from school discipline to school finance. With limited time or 
incentive to explore the complexities of the role of social research in 
restructuring the way problems are framed, it is not unreasonable for 
Congressmen to expect to see the NIE trying to solve pressing problems 
of education as they are perceived back home. The inability of the NIE 
to communicate with key education organizations in the early years, to 
involve organizations in a continuing way in planning, and to articulate 
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a program to educators in terms that sounded reasonable, all con- 
tributed to the perception of Congress— held perhaps until recently— 
that "no one back home cares whether this Institute lives or dies." 

In addition to these more typical kinds of legislative-executive 
relations. Congress is playing another more direct role for NIE. As 
the Institute seeks authoritative statements on which to build prior- 
ities and programs, other acts of Congress are assuming ^ew importance. 
For instance, the Special Projects Section of the 1974 ESEA renewal 
mentioned a number of particular areas of education that needed atten- 
tion; likewise the Congressional charter for a 1977 White House Con- 
ference on Education mentions specific items of substance. These kinds 
of Congressional statements are now being taken as indications of 

national priorities" which should play a part in NIE planning for 
R&D. So, in a general move to be responsive, the NIE is taking on the 
substantive agenda of successive Congressional education enactments. 

Beyond specific NIE matters altogether, it is clear that the 
Congressional climate of the day or term is an important factor. 
Senator William Proxmire's attack on social research at NSF, the 
recent demand that NSF stop its involvement with certain curriculum 
projects which some parents and Members find offensive to traditional 
values, the further legislation requiring NSF to submit for Congress- 
ional approval in advance all g 'ant actions, and so on, together form 
a climate that is bound to have a chilling effect on an agency doing 
similar work. 



The NIE Staff 

It would be most unwise simply to catalogue external forces 
pressing on Institute policy, as many important decisions and plans 
are in fact in the hands of staff, and their own values and concerns 
are therefore weighty. 

As a rule, the staff is younger rather than older; drawn from 
universities or government rather than public education, critical of 
much current educational practice rather than satisfied; anxious to 
get on with their own work and plans rather than perfect the plans of 
others (either plans made in the past, as in the case of inheritances, 
or plans made now by outsiders); and divided about the proper mode of 
attack on education matters (through basic studies of underlying 
processes, through field-based experiments and development, or through 
the spread of current best practice and findings). Many feel they 
were promised certain scope or freedom of initiative when they were 
recruited in the first heady days, and some would say the promise has 
never been delivered. 

Enthusiasms abound, but managerial talent and planning skill to 
carry them out is varied — as it is anywhere — so that not all staff 
feel well-organized and well-used. It is not surprising that the 
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Civil Service Commission has found flaws in the loose structure 
of the Institute and the frequent shifting of people from task 
to task— which are at variance with accepted government organiza- 
tion and personnel principles. The result will be a more tradition- 
al organization and hierarchy of groups, branches, sections, and 
divisions, which will be at odds probably with such staff values as 
flexibility and which many who are new to government will find in- 
hibiting and contrary to initial expectations. 

A former key NIE planner has stressed to us the combined effects 
of youth, desire to make a mark, and inexperience at planning and 
managing. As a result, the first several years saw a series of 
struggles among camps, each somewhat unreflective and not well-tied 
to outside advice and agenda-setting, but firm in the interest of 
pursuing a chosen tack — whether close studies of basic processes, or 
more practitioner-oriented work. The Institute's fledgling Office of 
Planning and Management understandably found it hard to bring these 
contentious parties to the table, and to consider in a mature and con- 
structive way diverse and sometimes conflicting efforts, though this 
pattern has been changing during the term of the Acting Director and 
with prodding from the Covincil. 

Thus, in the catalogue of forces, it is important to note that 
staff just as much as outside groups differ on what is best to study, 
with what methods, by whom, for whose immediate or future benefit. 
Staff differ, too, in their self-perception. Some wish to be "cata- 
lysts" or synthesizers, bringing together the best minds of the 
research world, drawing an agenda from them and proceeding with 
those parts of the agenda that funds and the state-of-the-art allow. 
Some others, perhaps still in hopes of recreating the type of activity 
they knew at the Office of Economic Opportunity, place more hope in 
directed work, field experiments, and close evaluation deslj^ned by 
the Institute to yield desired knowledge and fill information gaps 
for policy-makers. Still others are pleased to apply their skill in 
answering questions set by Congress in mandated studies such as on 
compensatory education or crime in the schools. 

The National Council 

Appointed late, amid Congressional criticism of the Nixon White 
House for the delay and the resulting difficult position of the 
Director, the Council has enormous formal powers. With the right 
not simply to advise, but to make basic policy for the Institute 
and oversee the program, the Council was seen by Congress as a way 
to insulate the agency from an untrustworthy bureaucracy and a 
balance-wheel in the chaos of fads in education. 

Since its swearing-in during July of 1973, the Council has met 
sixteen times (though not always with a quorum), and has come together 
in ad-hoc sessions for visits or discussions with other groups many 
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other times. This is a rapid education process, and one could wish 
that the terms of office were somewhat longer so that the experience 
gained by Council members would be available longer. And yet, despite 
this admirable record of meeting and conferring, the Council remains 
a rather invisible force, certainly not known among NIE staff as a 
source of policy that affects their lives; nor in the wider R&D or 
education worlds is it readily acknowledged, we sense, as a shaper 
of the young Institute. 

The Council, we understand, feels that that it cannot, and indeed 
should not, act without close relationships with the Director, that 
their proper role is to press for staff analysis of options, honest • 
reports on outside views, and for recommendations from the Director. 
The suggestion, by CEDaR, that the Council needed its own staff to 
counter potential biases or special pleading inherent in NIE staff 
work, was rejected summarily. In fact, the Council has gone on 
record expressing satisfaction with the present arrangement, in 
which one NIE staff person gives substantive help to the Council and 
several others provide logistic support. (Even within the Council's 
preference not for independent staff work, but for assistance in 
coordinating the work of NIE staff in policy-development, there seems 
to us to be full time work for several people.) 

How has the Council been an influence, then? There have been 
three ways. First, continuing stress by the Council on its needs 
for options to review, and continuing pressure for coherent presenta- 
tions by top staff of how directions were being set, have had strong 
impact on internal NIE processes. Staff who have no idea what the 
Council does will nevertheless recall a much more formal and elaborate 
planning sequence leading up to the 1976 budget, than had ever been 
seen before. This systematization was in large part a direct response 
to pressure from the Council. 

Second, and contributing to the invisibility even more, is that 
the Council has influence through a few top staff members who 
know them most intimately, as these NIE leaders internalize the views 
of the Council. Thus, without hearing the Council explicitly resolve 
upon something, a Director or other top official may himself reject a 
course of action because "I know the Council won't like that," or pro- 
ceed in a certain way ''because it's consistent with Council thinking." 

Third, of course, the Council does make substantive decisions, 
such as setting priorities and reviewing budgets. They opposed expan- 
sion of the voucher demonstration activity, for instance. They directed 
that 10-15% of the budget be allocated to field-initiated basic re- 
search. The 1976 budget received extensive scrutiny, and various pres- 
sures for and against programs were explicitly x^eighed and resolved. 
Yet even here, in the last two years, we are told, there have been times 
when private Council decisions have been announced publicly as staff 
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recommendations, then endorsed by Council resolution, so the source 
of policy is not fully clear. 

Examples of Council action on R&D funding policies include: 

® December 1973: Establishing five priority areas for 
future emphasis, and allowing the Director to defer to 
another year some obligations to free up funds for new 
work in the priority areas. 

® January 1974: Making clear they expected NIE staff to 
solicit public opinion as a normal part of the planning 
process and to reflect this opinion in material presented 
to the Council — in contrast to the isolation and non- 
consultation they perceived. 

^ September 1974: Obliquely endorsing the direction of 
diversifying the set of R&D performers, by supporting 
the Director in trying to bring current work to an end 
with useful results in the current year (Fiscal 1975) 
and urging that planning for FY76 give particular atten- 
tion to helping institutions align themselves with NIE 
program purchase policies. 

October 1974: In anticipation of Congressional action 
cutting the FY75 budget, the Council endorsed the idea 
of reducing lab and center work by 15% overall, based 
on project-by-project review. 

^ January 1975: Stressing that policy goals used by the 
Council in reviewing the 1976 budget also be used in 1977 
plans — including targeting funds to state and local agencies, 
stress on competitive processes, public involvement in 
planning, and programs responsive to specific needs of 
education. 

® May 1975: Directing that the Institute strengthen its 
emphasis on evaluation of R&D products and the capacity 
of R&D performing institutions, and asking that the Director 
return with some plans for action along these lines. 

The last example is of particular interest to us, in view of our 
own strong views concerning ways in which NIE should be more aware 
of institutional capacity than it has been. 

In some other ways in recent months the Council may be organizing 
for more effective action. We note that it has formalized its staff 
office, and in addition has set up committees of the Council 
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for specific functions. An Executive Committee will give continu- 
ing oversight to NIE implementation of Council policy, and will 
provide a manageable forum for considering new policy initiatives 
to be recommended to the full Council. A committee on Program 
Development and on Strengthening Education R&D can look at the 
substance of the various NIE activities and consider whether the 
mix of activity is responsive to continuing education problems. 
This group, as well as the third, on Council Reports, could be the 
forumis for consideration of the analytic work to be done by the R&D 
System Analysis unit we later recommend NIE should establish. 

Since we argue that the required reports should be seen as im- 
portant opportunities to tell the story of education R&D, we are 
glad to see the Council formally establishing a group to attend to 
these tasks. 

Regulators Elsewhere in the Executive Branch 

The forces we have been talking about have impact on both pro- 
cedure and substance, such as the Council in effect directing both 
the process and nature of plans by the questions it asks. There is 
in addition a set of forces acting on purely the procedural conduct 
of NIE business which concerns many staff, and may be worth comment- 
ing upon briefly. 

Outsiders to government often suspect the "interference" of the 
Office of Management and Budget (OMB) , the planning and analysis of- 
fice chiefly concerned with the President's annual budget. The agency 
has grown in recent years to have oversight of many more aspects of 
government activity than the budget, in the name of coordination. 
Typically, however, OMB has not intervened in the Institute budget 
process as it was famed for doing in earlier years of the 1970s in 
cutting other social programs; NIE is seen as an administration 
initiative, though not to justify an enormous battle with the 
Congress. The "forms clearance" procedure, carrying out the require- 
ments of the Federal Reports Act that all data-gathering instruments 
be screened by OMB5 is often decried by researchers as potential 
government censorship, but in fact seems not to be so exploited by 
OMB. 

Rules and regulations concerning employment, advancement, and 
organization of staff are made by the Department of HEW and the Civil 
Service Commission, and these, too, are seen by many at NIE as inhibit- 
ing and irrelevant. As mentioned above, staff have recently been re- 
organized, and much more exacting and rigid job descriptions have been 
ordered for each employee, in the wake of a critical review by the 
Commission of past personnel policies. Staff perceive that the Com- 
mission views all agencies alike, and would staff and organize a check- 
writing bureau in the Veterans Administration the same way as a re- 
search team at NIE — with obvious dysfunctional consequences. Whatever 
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the merits of the argument, the fact remains that NIE's hiring 
authority for certain positions has been suspended, the Institute 
has another black eye, and flexibility of action is reduced for 
the future. 

The general political climate of suspicion of education- 
related and research-related grant and contract procedures, 
suspicion encouraged for years by now-retired Representative 
Edith Green, leads many business administrators in the Department 
of HEW now to interpret procurement regulations in the strictest 
ways, we are told, with the result for instance, that NIE seems to 
be virtually precluded from considering unsolicited proposals that 
are not part of some formal competition. The episode last year of 
a disgruntled contracts officer from NIE publicly spreading a 
mixture of fact and fancy about "scandals" in NIE procurement on 
radio, to Jack Anderson, and anywhere else he could find an ear, 
with the inevitable resulting General Accounting Office inquiry 
lasting for months, has had a chilling effect. 

It is increasingly difficult to involve groups of outsiders 
formally in the planning and review of the Institute's program, 
some feel, just as that becomes a more central tenet of policy. This 
problem results from a Federal Advisory Committee Act and additional 
Department policy which lay strict rules on officially-constituted 
advisory panels, and conflict-of-interest rulings by NIE and Depart- 
ment attorneys which make it very hard for anyone associated with 
research to declare him- or herself free of possible benefit from 
the advice to be given. 

From the parent Department of HEW itself, including the Secretary, 
the various staff offices serving him (Planning and Evaluation, Budget, 
Administration and Management), and the Assistant Secretary for Educa- 
tion who presides over both NIE and USOE, there seems not to have been 
any remarkable degree of general programmatic regulation or oversight. 
Again, as an Administration initiative, the NIE has been an object of 
concern, dismay, solicitude, but no constant or overt meddling (nor 
grand gestures of pride, pleasure, or support). The Secretary and the 
Council jousted over expansion of the voucher program, with each finally 
tacitly agreeing they could veto the other, so a compromise was worked 
out between the Secretary's eagerness and the Council's reluctance, to 
fund some further planning. But aside from that particular episode, 
however upset the Department may have felt at either the substance of 
the NIE's work, or its political ineptness, little action has ensued. 
We do conclude that the new Institute has needed flexibility to operate 
in areas such as procurement, which has not always been understood or 
sympathized with at the Department level. And we urge that for both 
that purpose and for others, the ties between the Secretary's office 
and the NIE be examined and strengthened. Strong support will especial- 
ly be needed from that quarter if the lab-center review process we have 
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in mind is to be successfully implemented. 

The Effect of Past Actions as Determinants of Present Possibilities 

A final feature of the context that we must note is the crushing 
history of inherited commitments that we hear about repeatedly from 
NIE staff. The Institute, they wish us to recall, did not start fresh, 
and was not able to build its program as it chose. It was "given" a 
great many staff and programs from USOE, OEO, and perhaps even else- 
where, despite the Congressional intention that NIE avoid many past 
mistakes of research management. Yet the early budgets were entirely 
filled up with such items, contractual commitments that had to be 
honored, of enormously varied quality and purpose, though to be sure 
the Institute had signed many of the contracts itself. 

Thus, in the first few years, goals could not be set a priori, 
performers could not be chosen against goals, nor new constituencies 
be imagined; all came with the territory — projects, project monitors, 
implicit and explicit goals, performers of R&D and constituencies who 
believed in the activities. 

If appropriations had risen as everyone expected, the inherited 
work would not have bulked so large. But the Institute's powers of 
persuasion did not match its appetite, and under a steady budget the 
weight of the past has been felt more and more. And, of course, one 
cannot build new constituencies when almost all funds are tenaciously 
being retained by old constituencies. The mood must have been schizo- 
phrenic in the first years, as NIE was expected to quickly demonstrate 
new directions and plans, but was also asked first to "show what you have 
accomplished with your present funds." 

We can understand, too, the seiifie of fairness which may have been 
felt to prohibit dif f erentiated defiingr^ with laboratories and centers 
in the early years. Three-year co'itracts had been signed in many cases 
for specific programs. To change course in mid-stream to a more insti- 
tutional relationship with some, even as several NIE programs might have 
wanted, was seen as impossible because of the general commitments made 
to a program-purchase policy. Even, so, this history hardly explains 
why there should be such dearth of policy thinging about the institutions 
now , at the end. of the three-year commitments. 

Conclusion 

From groups. Congress, staff • and past commitments, then, the NIE 
has available not so much a mandate but a contradictory (and often 
mutually exclusive) set of directives as to basic goals, means to attack 
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them both of method and performer, rights to participate in the 
process, and expectations for the eventual result against which 
to measure achievement. And from the standpoint of basic re- 
sources, Institute managers have scrambled to preserve some in- 
ternal freedom to initiate in the face of heavy aggregate inherit- 
ances and heavy outside pressure to constrain the budget through 
earmarks . 

In this context, the one basic and fundamental dilemma of 
research managers that we do not envy is resolving the competing 
pressures for scientific activity vs. constituency-building. Since 
the political muscle of the organized basic science community — 
those for whose support and nurture the Institute was in part cre- 
ated — is tiny, the alternatives are closer ties to the concerns of 
the service-delivery system and its powerful constituent associations 
and/or the organized R&D institutions in groups such as CEDaR. Both 
such moves have definite consequences for the range of acceptable 
activities the Institute can support. If the burden of making up a 
politically viable program each year seems too impossible under this 
set of mutually exclusive pressures, it may become more and more 
attractive to think of funding research through "set-asides" or 
permanent legislatively-earmarked fractions of the funds appropriated 
for educational service programs. For instance, research on comp- 
ensatory education could be funded handsomely with a fraction of a 
per cent of the appropriation for Title I of ESEA. Of course, 
pressures would still be brought to bear on actual spending decisions 
made by the agency, but at least the basic annual funds might be less 
vulnerable and problematic. The constraint in this case would be 
that the Institute's activities would be much more closely tied to 
the substance of legislative enactments of the Congress. 

Within this set of forces, then, NIE is taking certain policy 
directions in its Fiscal 1976 budget, and in the 1977 plans now in 
process. These directions include focus on various tasks or problems 
to be done in various ways by various personnel, and procured in 
various fashions. We turn next to the R&D funding policies that are 
our major focus of assessment, analysis, and recommendation. 
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This consultant group was convened out of concern at NIE to 
have an outside look at its policies towards a set of institutions — 
the regional laboratories and the R&D centers. In our view, the 
direction for such institutions should result from two sets of anal- 
yses and policies: on the one hand, geheral NIE program planning 
and design — choosing substance to work on and making further choices 
as to strategy; and on the other hand, close understanding of the 
various resources available to carry out the kinds of work implied 
by the planning process. With these two kinds of information well 
in hand, policies of procurement can be tailored to fit the state of 
the community of performers who might be capable of doing the job, 
and subsequent policies of management can similarly be fashioned 
depending on the work and the worker. General is sues at the level 
of "for or against institutional support," or "loose or tight moni- 
toring," must be raised in more concrete contexts. 

The following discussion focuses first on the current Institute 
policies of program design, planning, and substance. Then we review 
the ways that resources seem to be found and linked up with the In- 
stitute's program. Third, we discuss specifics of the funding pol- 
icies toward laboratories and R&D centers. We conclude this section 
with a discussion of a new role NIE should take on, of understanding, 
analyzing, and giving leadership to the whole R&D system in education. 

We should say at the outset of this review of NIE policies that, 
as noted in the preface, we emerge from our work stronger supporters 
of the NIE concept than we were at the start. We believe in the idea 
of such a lead agency, and we appreciate the efforts of staff, grant* 
ees, contractors, and outside groups to keep the agency going in the 
first three years. However we do not accept or support NIE completely 
as it is, and we see a number of essential changes in policy, practice, 
and general outlook that must take place for NIE to fulfill its pro- 
mise. 

We, probably along with many other distant observers, had been 
aware before taking this assignment mostly of the bad news, the poor 
publicity the NIE had been receiving in almost every quarter. Yet 
with the opportunity both to look more closely at some of the com- 
plaints, and to learn a great deal more about the Institute's staff 
and directions, we find much in the recent months to feel positive 
about. 

° Begun by the Acting Director and continued in the 
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recent weeks by the new Director, a serious arid 
sincere effort to strengthen relations with various 
constituencies, Congress, the teaching profession, 
state departments of education, and others. 

° Greatly improved responsiveness to Congressional in- 
tent, in the presentation of the Institute's program 
and in carrying out mandated studies. 

Serious strategic thinking done in preparing the Fiscal 
1976 budget, with hard choices made and priorities set. 

Revision of the agency structure to emphasize substan- 
tive areas of work, rather than functions such as re- 
search or program development which have little meaning 
to the general public. 

Establishment of new committees by the Council for more 
extensive work, and an Executive Committee for further 
work between meetings of the whole. 

° Opening of NIE planning and policy-making more and more 
to the scrutiny of outsiders, such as with our own group, 
or the group which spent two days reviewing tentative 
program plans for Fiscal 1977. 

° An increase in internal communications through newsletters, 
more frequent colloquia, seminars , presentations by guests, 
and other means, to reduce the isolation of programs from 
each other and rebuild a spirit of productivity among staff. 

Clarifying to some extent internal planning and decision- 
making processes. 

Some cv>mplaints from the field seem sometimes to take lit::le note of 
these things. We have occasionally felt we were hearing anecdotes from 
the first year or so of the agency, told and re-told without charity, 
historical perspective, or recognition of the present shifts. 

Planning and program design 

There are three policies of this sort that we turn to first, as they 
result in the substance toward which funds are directed. They are: 
defining the agency's purpose in terms of work on six specific education 
problem areas; involving a wider range of groups, individuals, and or- 
ganizations in setting the agency's agenda; and better balancing of the 
actual activities supported, between payoffs in the long, medium, and 
short run. We take them up in order. 

Problem focus . In part reflecting "priorities" identified in 1973 but 
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not emphasized in budgeting before FY76, the six problem areas chosen 
as the budget and organization structure in late 1974 are as follows: 

° Improving the connections between education and work; 

° Finding ways to remove barriers to education caused by 
racial, language, or sex prejudices or stereotypes; im- 
proving the equity of treatment of participants in educa- 
tion generally; 

° Improving the capacity of schools, school districts, and 
community groups to solve their own problems through in- 

• quiry and innovation; 

° Improving the productivity of education through strengthen- 
ing financial, management, and technological tools; 

° Managing an increasing base of data on research, products, 

and good practice, in order to disseminate it, and helping 

states and school districts to gain access to the base and 
link it to users; 

° Strengthening the ability of the schools to teach basic 
skills. 

We support strongly the problem-focus and regard the problems 
chosen as significant and needing attention. We find such support 
rather easy, however, since we share the perception found many times 
over in the field, that the areas are "broad enough to 'irive a Mack 
truck through" and can encompass most of the major education issues 
one could conceive of. (If anything, we would welcome even sharper 
definitions of NIE targets, based on assessment of areas where the 
strongest impacts could be made with limited funds.) But it may be 
very useful to have such foci built in for a while, to be able to 
answer the endless petitioners for funds for every imaginable scheme 
that "our appropriation does not cover activity of that (other) sort," 
and to be able to tell Congreas just what the NIE is up to. And as 
presently-supported insti^utions wish to know about their future role 
in the NIE program, it can now be clear that major funds will be spent 

• only on activities within those problem-areas and that no other 
fields of work can be supported to any substantial degree. In the 
climate of tight money and demands for clarity now pressed on NIE, 

• ' we see no alternative to such a policy of focus, no way that NIE 

could spend major funds, for example, simply according to scientific 
disciplines (a program in sociology or a program in psychology, for 
example) • 

We are concerned about the substance of the chosen areas as well. 
For it seems to us that the problem-focus of the 1976 budget may not 
in fact reach key problems. For instance, it is not clear to us what 
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the overall NIE effort will be in the areas of education of the 
disadvantaged and problems of inner-city schools, either in pro- 
gram content or resources. At least none of the budget materials 
and analyses that we saw allowed us to piece together from the 
present problem-focus how activities would relate to these special 
needs. As another example, we might ask about concentration on 
certain elements of the educational program in the country, such 
as the junior and senior high years, which have been the focus of 
considerable debate at a highly abstract level in recent years but 
which could be followed up in concrete plans and designs with profit. 

In view of our concern about gaps in the coverage of present 
problem-areas, it seems to us that iu developing plans for work 

within each problem-area, some more formal advisory or intelligence • 

system could be at work to insure that the definition of the problem 

stays ''real" and current with the state of the education system. 

Perhaps some common "health of education" indicator projects with 

the National Center for Education Statistics could be mounted to 

keep an eye on broad trends in each problem area. 

We agree with the Council that NIE should not be held account- 
able for solving each of the problems it identifies for attack — 
education and work in the present economic situation is a clear 
case. But we sense that there is no long-range planning process 
either for changing the mix of activity within a problem-focus, or 
revising the overall set of problems. Most other agencies have 
equally, if not more, primitive processes for planning social R&D, 
so we do not mean to imply some high standard that NIE has uniquely 
been unable to meet. And indeed, the creative adaptations by sev- 
eral NIE programs of the National Cancer Institute planning process 
are to be applauded as very useful experiments which deserve study 
in their own right. However, for the future, and especially above 
the program level, we sense that there should be an equally creative 
planning function, beyond that for crisis analyses and annual budget 
reviews. Perhaps there should be a separate staff of two or three 
people within the planning office who are well-read and catholic in 
their interests and contacts, with a few resources to convene people 
outside of current program areas. Hopefully such a staff could listen 
closely to users of R&D, to bring back to the NIE not what can be done, 
but wliat should be done, when the time comes to reconsider the prob- 
lem areas for NIE focus. 

Broader involvement in setting the agenda . The Institute has a 
policy of now seeking broad participation in the design of its work, * 
and such a claim is regularly made about the development of the chosen 
problem areas. But the choice of such broad headings is really only 
the beginning, and the subsequent steps of choosing sub-priorities 
and strategies need to be as open in process, and as credible in sub- 
stantive result, as the first step. We find concern in the field on 
both counts. 
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As to process, we heard repeatedly of anger at unmet expectations 
concerning the openness of planning* Scholars and practitioners, in 
associations and institutions, told us of promises made and broken, 
"involvement" that was neither serious nor sustained* A teacher group 
especially forcefully reminded us that occasional meetings in Washing- 
ton of high-level staff are no substitute for serious participation 
of practicing teachers in the work of NIE grantees and contractors 
from the outset. With all the talk about a new emphasis on outside 
views at NIE, it was disturbing to hear that any group needed, as 
these teachers did, to "break down the door" to get to the table at 
NIE planning conferences of significance to them. 

We heard that *the problem areas were chosen in part because they 
covered a significant fraction of the present work NIE has under way. 
But if this is true, it is further disturbing — again on the subject of 
the process of planning — to hear as we did repeatedly at laboratories 
and centers, as well as at other R&D institutions, that top-flight 
staff at such places with extensive background in relevant work, feel 
distant -and uninvolved in the planning and designing of future work in 
their areas of competence. 

We recognize that pex^ple who complain about being uninvolved may 
in fact be seeking only approval for continuing what they are present- 
ly doing, rather than analyzing a situation and entering into a plan- 
ning process. We recognize the several agenda-setting conferences 
that NIE has held, with massive participation of many interested groups 
and individuals. Thus, we cannot judge the factual accuracy of these 
claims about participation. But even the perception in the field — 
not confined to the beleaguered labs and centers, though as appropri- 
ate for them as anyone else — that NIE is aloof and hostile, will be a 
continuing problem in constituency-building and drawing on the full 
capacity of the R&D system for NIE work. 

Our own contacts with the NIE staff have shown them to be bright, 
capable, hardworking? and far from the stereotype of the Federal 
bureaucratic timeserver. However, we feel that their very articulate- 
ness, and their concern to be involved with the work they oversee, 
may appear overpowering and arrogant. Whatever the shortcomings of 
research management at USOE, the research field may now look back on 
those days of laissez-faire as a golden era of non-interference by 
the government as many new employees at NIE carry out a more direct- 
ive style of research management, and not always with the research or 
practitioner credentials to legitimize such a role. 

Reducing long-term development commitments . As a third element 
of planning and program design, we note a policy at NIE aimed at 
balancing long- and short-range activities more equally in future 
budgets. NIE argues that the mix of activities has in the past been 
too heavily weighted by large-scale, long-term contracts for "develop- 
ment" work, chiefly at laboratories and R&D centers. The NIE argues 
in its 1976 budget materials that program plans reflect a move away 
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from such a concentration, to allow for shorter-term payoff activity 
in dissemination and in policy-studies for decision-makers who need 
information soon, and to allow for reinstatement of new basic research 
awards which were cut off in 1975 when Congress reduced the overall 
budget. 

These new directions may have several positive features — politi- 
cal appeal, substantive merit, and responsiveness to Council policy. 
But we have reservations about this policy, to the extent we under- 
stand it. Either we do not grasp the reasoning against "development" 
in general, or we do not see distinctions among types. So far as 
developed products from R&D centers and laboratories chiefly in- 
tended for teachers and children, we did not gather new data on 
quality and impact, nor was NIE's own survey data on products yet 
available, for us to judge the quality of this type of work. Several 
of us who pressed staff of labs during visits or in correspondence 
concerning effectiveness remain unimpressed, and we understand that 
the National Council shares the general view that contractors have 
not had funds or other incentives to gather extensive, long-term data 
on product impact. Yet we liked what we saw at a number of places, 
where unique and useful materials seemed to be under development and 
dissemination.- A number of our correspondents agreed, and our talks 
with leaders of the National Education Association, American Associ- 
ation of School Administrators and the American Educational Research 
Association showed further consensus that s ome places were doing do- 
ing development work of great promise or actual accomplishment. 

We recognize that some NIE problem areas will continue to support 
lengthy development activities of various kinds. And we recognize 
further that it is quite proper to question the continuing need for 
large-scale curriculum projects, for the vogue of such efforts has 
somewhat passed and te^^chers are often asking for different kinds of 
help than just through materials and media. Also, publishers have 
increased their output o£ diverse and attractive materials, some 
even as the result of field tests and including built-in objectives 
and assessment devices, all of which used to be the more exclusive 
hallmarks of Federally-supported laboratory products. 

However, there are other kinds of "development" that seem to us 
to have good claim on NIE resources, and which should not be over- 
looked in a rush to get out of an expensive type of activity. We have 
in mind, for example, development of models of in-service teacher train- 
ing; system-change development work, such as rethinking compensatory 
education altogether as NIE has been asked to do by the Congress, but 
which could have been initiated by NIE if it had not been asked; dev- 
eloping post-elementary school models that incorporate some of the 
thinking of the past few years about the fallacies of the comprehens- 
ive high school; developing alternative concepts such as the employer/ 
experience based career education projects; and developing new decision- 
making and communications systems between school and community, or 
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between students and administrators. These are just a few examples 
of education program development , beyond curriculum materials, that 
seem worth exploring, and we hope that there has been no decision 
against such projects. We of course understand the need for having 
work under way that will result in tangible results in the near 
future as opposed to projects requiring four or five more years. 

As a final note on program planning and design, we understand 
that NIE expects to reinstate a policy of supporting basic research 
in various problem areas to a total of approximately the Council- 
mandated 10-15% of the budget. We hope that NIE in so doing will 
not return completely to the earlier pattern of scattered, uncumu- 
lative single-project work by university and college scholars. 
While some funds should be available for promising scholars who 
have not managed large projects (or do not wish to), we also feel 
that investment in longer-term efforts and sizable groups should 
be encouraged. Under that pattern, . social systems can form, the 
group can grow and develop, adapting its approach as the problem 
becomes better known, and training some students along the way by 
apprenticeship. We do not underestimate the difficulty of getting 
useful research out of groups of university faculty, and as we noted 
above, the R&D center must be only one among many strategies for 
attempting this feat. Nor do we place sole reliance on limited 
studies by a professor and an occasional helper or two. We hope 
that as NIE seeks "better balance" in the program mix, grants of 
any size will be made after serious thought as to the state of 
particular fields and the potential for significant advance through 
the precise type of award being considered. 

Finding and Supporting Those Who Can Best Do NIE's Work 

We understand there will be in effect in 1976 a policy to diver- 
sify the performers of NIE's R&D activity, with open competition as 
the strategy for this widening of the net. We agree that there are 
untapped resources in university departments including education or 
in the publishing industry and underused resources in local and state 
education agencies. But we also have concluded that there is not any 
excess or over-abundance, or even adequacy, of R&D resources, especially 
not resources of high quality. So we feel it is simply not true that 
one needs only to advertise the NIE's needs in order to have immediately 
an array of qualified people or organizations to choose from. We dis- 
cuss two aspects of linking up with available resources: first, knowing 
present work, and second, drawbacks to the present style of competition 
practiced at the Institute. 

Hoping to find some new friends, new constituencies for R&D, and 
hoping also to start fresh projects that could avoid the discredit of 
the old, it seems likely that NIE staff in its first two years had 
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incentives to view inherited projects and their performers as unim- 
portant, of inescapably lower quality than anything new. Indeed, the 
presumed shortcomings of the past research establishment had been a 
chief reason for wanting a new agency. Thus the familiar "not invented 
here" syndrome may have "justified" not giving inherited work close 
attention or investment of time in review and management. Recall that 
budgets were to rise, which meant large plans on the drawing boards, 
consuming most available staff energy. Some of us heard repeatedly 
from project officers that "no one had taken a very close look at 
the work on this project in the first year or two after we got it." 
Such comments were mirrored in our field visits and questionnaire 
responses, as managers of work at laboratories and centers recalled 
their projects being shuffled from office to office, with no project 
monitor in place long enough to become familiar or effective. 

The situation seems to have improved, and these same project of- 
ficers in most cases had been with their charges for some time, though 
travel allowances to be with projects in person seemed inadequate for 
some of the larger efforts. But it troubled us most that individuals 
clearly were operating iit a vacuum about how to work with projects, 
how closely to be involved, how much they needed to know to carry out 
what sort of role. Such a situation, of extreme diversity in how much 
is actually known about specific work in the field, especially at lab- 
oratories and centers, makes one skeptical about policy changes that 
should rest on judgments of present activity. 

Our second concern about the way NIE uses its resources is its 
apparent policy of procurement by contracts awarded after open compe- 
tition in all but exceptional cases. 1. (We are aware that in the last 
two years, owing to tight budgets, many procurement actions were sim- 
ply continuations of grants and contracts awarded at the outset. Nev- 
ertheless, the intended policy direction for the future, and for chose 
new funds let out in the recent past seems to — and to lu^ay in the 
field — to be towards open national competitions, notwithstanding sever- 
al exceptions cuch as tha Problem- Solving competition that we were told 
about repeatedly.) This emphasis on competition was recently reinforced 
in the Senate appropriations report for FY 1976, and was put forward 




NIE's own data show a preponderance of funding through contracts. 
In FY74, 497 contract actions accounted for $47.6 million; 129 grant 
actions accounted for $14.6 million. Through April 1975, 204 contract 
actions resulted in award of $44.4 million, while 18 grants had re- 
sulted in $5.4 million of awards. The choice of instrument has many 
interesting and important ramifications that we do not go into here, 
but which need exploration in the NIE context - including implicit 
incentives to use contracts because of the 10-15-month delay in issu- 
ing regulations for grant awards, even though such awards can be made 
with much more flexibility of procedure than contracts. 
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strongly in the Background Paper and in other NIE documents. It 
may be politically important just now to have a policy like this 
to counter lingering criticisms of U.S. Office of Education procure- 
ment of social research. But we feel it has serious drawbacks, 
both intrinsically and as presently used, as a tool for any particu- 
lar deliberate action such as diversifying the performer mix. 

As typically executed, the open competition based on an agency's 
Request for Proposals (RFP) has at least the following problems: 

It is not always true that agency staff can write clear 
and useful specifications for what is wanted, particularly 
if the work stems from a planning process where the agenda 
was drawn up by national experts from outside the Institute, 

° The costs of bidding are eventually added to the govern- 
ment's cost in future procurements, so a high rate of 
bidding and the accompanying high rate of unsuccessful 
proposals is in the long run drawing funds away from 
performance of the work. Where there are only a few 
good performers for a given type of work, the rest of 
the competitors have little chance, and their costs of 
failure are a drain on energy and time that might have 
been avoided. Further, the cost to the government of 
reviewing a great many proposals is not always reflected 
in superiority of the final product as compared with the 
quality obtainable under more limited competition. 

Extensive competition among a small number of organiza- 
tions capable of large-scale work in education R&D may 
tend to promote disintegration and professional secrecy 
within the group — negative- results to be avoided if 
possible. 

° Where proposals are judged by Institute staff, the current 
procedure prevents them from working with proposers to look 
at ideas or to review advance copies of proposals so as to 
avoid submission of obviously unresponsive or unqualified 
ones, or better, to strengthen marginal ones. 



Researchers who examined the competitive process in a slightly dif 
ferent field, social program evaluation research, commented how compe- 
tition failed to produce quality, and in fact quite the contrary. 



■*'A. Biderman and L. Sharp, The Competitive Evaluation Research In- 
dustry (Washington, D.C.: Bureau of Social Science Research, 1972), 
p. 46. 
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In addition to the above-mentioned barriers to quality, and 
hidden costs, the strategy of open competitions as a device for 
bringing to the attention of the NIE all those who should be con- 
sidered for research funds rests on questionable assumptions about 
the market. First it is assumed, explicitly in the Background Paper 
on "Labs and Centers ," that there are now abundant resources for edu- 
cation R&D as a result of the heavy demand for social science evalua- 
tion and advice in the 1960s. We feel, on the contrary, that this is 
a misplaced hope, and that a few hundred consulting firms do not an 
R&D system make. Second, reliance on competitions assumes that those 
who would be good performers know about the solicitation and are able 
to enter the market. Some challenges to this assumption, at least 
based on present NIE practice, must include the following: 

° Little or no advance information about specific competitions 
is available except through personal contacts, which effect- 
ively confines participation to those with well-developed 
private intelligence systems or those already so aggressively 
in the market as to scan the official Federal procurement 
publications. Specific competitions open and close on very 
tight schedules, leaving little time for informal contact to 
spread the word. The process of advertising contract pro- 
curements is especially cumbersome and ill-suited for reach- 
ing the academic community. 

Nor does any general information at a decent level of speci- 
ficity exist concerning the overall thrust and scope of each 
program, so that a person could review planned activity and 
inquire further how to participate. 

No single document is available such as a monthly list of 
present competitions or RFPs available^ (though we recognize 
that fewer new competitions were held in the tight budget 
year of FY 75). 

Mailing lists, we understand, have been a continuing source 
of difficulty. Apparently the Institute does not have a 
ready way to idencify people who have indicated interest 
in the past. 

No general information exists about the mechanics of grant 
and contract application and award procedures, such as a 
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general guide to "How to get funds from NIE/' Even 
small basic research projects are announced and awarded 
through the formidable method of the RFP and the contract, 
without guidance as to the meaning of the various parts.^ 
to be completed, the full process that will be followed in 
making decisions, rights of debriefing, and so forth. The 
perception is widespread in the field that doing business 
with NIE is extremely complicated* and it is probably right. 

We understand that so inadequate is the Institute's general publica- 
tions effort, some staff have been reduced to sending out xerox copies 
of the staff newsletter or the interim telephone directory for lack of 
anything else timely, comprehensive, and informative. 

To be sure there are exceptions, and we have seen isolated plans, 
concept papers, or documents from programs listing active grants and 
contracts. And a newspaper-format publication has begun to issue in 
the last year, though it focusses on isolated projects, not on the In- 
situte itself, and is not aimed at the R&D community. Even staff mem- 
bers on the Institute's authorizing committees on Capitol Hill shake 
their heads that a place with such a need of public understanding, 
and with presumably some expertise in communications and teaching, 
has such meager ways to inform the professional community with which 
it must work to carry out its program, let alone educate the broader 
world about its activity. 

In short, we doubt that NIE R&D opportunities are as widely 
known as they should be, if one is to put confidence in open compe- 
tition. And when we add in all those who now avoid NIE because of 
shabby treatment in the past (proposals lost, advice asked at hastily 
called convention conferences, red tape of grants management), and 
the others still who would be superb if joined with one or another 
across the hall or down the street, we begin to add up a large segment 
that may not be fully engaged by the present process. We are not 
surprised to hear NIE staff comment after procurements, "We were 
surprised who bid and who didn't," or "We just didn't get the quality 
of proposals we hoped for, but we had to go ahead.... 

The impression left in the field, which we tend to share, is that 
reliance on open, national competitions under the procedures now used 
by the Institute can be in part a rationalization for being ill-informed 
about present performers and their capacity to continue present work 
or redirect themselves to new areas and for avoiding the substantial 
task of identifying and being in touch with ' the diverse segments of the 
R&D system that should be brought together to play a part in NIE's work. 
We recognize the dangers of simply allowing each program manager to 
award funds non-competitively , and we agree that review processes are 
useful and necessary. But the costs and inadequacies of the present 
completely open competitive methods are all too plain as well, so we 
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conclude that procurement management is a prime area for additional 
staff, research, and innovation.! We present a number of recommen- 
dations towards that end. We are simply disheartened to see the 
weak links between the Institute and the community of R&D performers 
after three years, and the apparent reliance on wasteful mass invita- 
tions for proposals to determine who is interested and who should be 
considered to do NIE's work. 

Regional laboratories and R&D centers 

Though commonly lumped together in language and policy as ''labs 
and centers," we find them hard to deal with that way, as they share 
little except the accident of transfer from USOE at the same time 
and their status as initiatives of the Federal government some years 
ago. The tables in Appendix C demonstrate their variety of budget 
size, staff, and degree of dependence on NIE. The narrative data 
they submitted further underscored the point, as we mentioned in 
Chapter III above. 

Laboratories seem to us a unique structure, poised between the 
university and the service-delivery system of education for a variety 
of purposes. Their shortcomings we feel are partly the responsibility 
of the Federal government for not seeing to it that events proceeded 
differently, and partly the result of the weak quality of some of 
their staffs and boards. University-based R&D centers seem to us 
only one way of organizing university talent, with certain drawbacks, 
and we favor use of various options as suits the work and the campus. 

So how does NIE policy seem likely to affect these organizations? 
First, what is the policy? Leaving aside the question of how NIE will 
respond to the conflicting signals from the House and Senate regarding 
earmarked funds, NIE policy towards laboratories and centers seems to 
include the following elements: 

Attempt to complete present products and terminate 
plans for further work by Fall 1975 when many three- 
year contracts expire. 

Where specific continuations are desired on specific 
projects,, individual program offices will arrange to 
present sole-source award justifications for Institute 
review. 

■"■Our own analysis of propurement issues and specific alternatives that 
should be explored could not be as extensive as we wished. Others have 
been working in this field, however. We refer to John G. Wirt, et al, 
R&D Management Methods Used by Federal Agencies (Lexington, Mass.: D.C. 
Heath, 1975); Volume 2, Part B of the Report of the Commission on Govern- 
ment Procurement (Washington, D.C.: GPO, 1972); and the active research 
which may be most useful of all, the National Academy of Science "Study 
Report on Social R&D.'' This last group is giving particular attention 
to procurement issues. 
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® Certain specific competitions may be designed to draw 
exclusively on laboratory or center expertise, in such 
activities as dissemination and product installation^ 

® Some promises are being' made about help in finding 
other funds. 

Encouragement is offered for institutions to enter 
any other Institute competition in the coming months. 

Lab and center heads tell us about deep staff cuts they antici- 
pate, because they are unsure of funds after this Fall. The tables 
in Appendix show that labs project 400 of 500 professional staff 
now supported 'jy NIE will have to be terminated for lack of other 
funding, and centers report a like figure of 200 out of 426. The 
questionnaires brought us pathetic examples of the grueling search 
for other work — includinp; re-norming tests for the Air Force and 
doing $1500 Title I evaluations. But NIE staff explain that over 20 
million dollars were to be spent at labs and centers in the coming 
year even before the House and Senate directed a greater amount, so 
that such estimates are not accurate reflections of how the full 
NIE policy will work out. Still, it seems unreasonable to us that 
just a few months before such possible serious impacts, labs and 
centers should lack information about what competitions will be held 
and when, or what sole-source awards they can expect. 

We could not review each institution or program and make specific 
judgments and recommendations, nor were we asked to do so. Our brief 
visits showed us places of high energy and appealing products, and 
others with more mixed report cards. Our correspondents agreed. We 
ended up feeling that no group of education R&D institutions per se 
deserves continuing Federal support, ovin5 to the g^eat diversity of 
work and activity present -in the field. 

But beyond that, we are simply appalled at the elements of policy 
that are not in place, the apparent disregard of the history of Fed- 
eral involvement with the labs and centers, the lack of deliberate 
review of the total capacity of each institution to ascertain its 
suitability for a continued role in the NIE program. If, as a gener- 
al principle, no institution is owed a living, and even if a number 
of present labs or centers could be closed without serious loss, 
nevertheless there should be some overall process of reaching deci- 
sions that can disrupt institutions and careers. The absence of such 
a plan of procedure is indefensible historically, substantively, and 
politically, and we strongly urge the Institute to avoid letting the 
aggregate of project-by-project decisions rule the final outcome. 

■''As a general matter, beyond the issue of evaluating lab and center 
work specifically, we find it surprising that NIE has no general 

(continued on page 58) 
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We can understand the history which brought things to this point. 
In 1972, when the new Institute was just starting and laboratories 
and centers changed their support method from a single contract to a 
series of project-specific contracts, each with its own timeline, 
milestones, and responsible government official, a shuffle ensued 
so that projects at a single lab or center have come to rest in far- 
dispersed comers of the Institute, subject to the idiosyncrasies of 
project monitoring that we discussed above. In fact, we discovered 
at the outset of our inquiry that NIE has no up-to-date information 
on labs and centers as whole organizations, "since NIE deals only 
with projects now." The rationale was no doubt convincing in 1972; 
the agency will grow, funds will be spread around and the laboratories 
and centers will be a diminishing part of the total group of NIE's 
R&D performers. in this view, special attention to these few places 
was not warranted. 

But of course the budget did not rise, and the labs and centers 
have remained a sizeable, if decreasing, fraction of the total Insti- 
tute budget. And despite the central role of people and projects in 
such places, the NIE continues to manage their work with apparent in- 
difference to the institutions as entities, and relates to them still 
through individual project officers except for confrontations at the 
top level. And with the sudden possibility that has arisen recently 
that some additional awards to labs and centers may have to be made 
in response to Congressional requirement, the Institute finds itself 
scrambling to consider what the places are good for — considering yet 
again the familiar device of the open competition to substitute for 
mutual communication and solid knowledge, saying in effect, "You tell 
us what you think you are good at' doing." 

So our major impression of policy with respect to labs and centers 
is that there is none where it counts most — namely in thoroughly can- 
vassing their likely contributions to NIE's work. The result seems 



(continued from page 57) 

evaluation group, policy, or support function. Thus each project of- 
ficer, each new writer of an RFP, each proposal reader, to some degree 
reinvents the wheel or expresses personal idiosyncrasies, in the absence 
of central guidance and sustained attention to the topic of criteria and 
standards of quality. Sam Sieber has some thoughts on this in his essay 
in Appendix A. We note that such a missing piece of activity, such a 
gap of intellectual substance, makes it easier for critics of decisions 
such as not renewing a certain lab or center contract to charge that 
politics or budgetary constraints are chiefly to blame, rather than to 
confront the possibility of bad work. It seems odd that NIE has been 
able to resist taking a sustained look at these issues, in view of the 
incessant pressure on local schools and other Federal agencies to be • 
involved with evaluation. 
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llkely to be the closing or changing of a number of institutions. 
While institutions with special relationships should be decreased 
in number and should have their missions clarified, uncoordinated 
actions are not the way to do it. 

Taking the goal of an institutional capacity such as we outlined 
in Chapter III, a small number of very high quality national R&D 
facilities in education, we feel when the set of such institutions ^ 
is in place, that they ought to consume no more than a third of NIE s 
budget. If we consider that a decent operating size should be at, 
least $3 to $4 million each, per year, it is plain that NIE can 
support only a fraction of the present number. A complete review 
of programs, finance, and management at each institution can result 
in a considered decision whether a place can contribute to the NIE 
problem areas and the degree of reorientation needed to do so. 
Only such a detailed review and holistic judgment can do justice to 
the Federal investment in building the institutions. We think such 
a review process should be done soon even if NIE's budge t were twice 
its present size. Where reviews indicate that a place will indeed 
no longer receive significant NIE funds, the concept of transition 
funds should be revived, to aid in shifting to other missions and 
funding sources, or to aid in orderly and considerate closing of 
the institution. 

Where institutional capacity seems strong and appropriate to 
NIE's problem-areas, we do not advocate returning to the USOE manage- 
ment methods or to some new variant of institutional support. We 
sense that organizations in the field do not expect this. We do 
sympathize with their hopes for stability of funds, but we would 
argue that institutions which seek such security must be willing 
to allow substantial NIE involvement in planning and carrying out 
the work and not expect to be given a large contract and left alone 
for years. Three-to-five-year awards should be made to the smaller 
set of institutions, for careful ly-worked-out activities oriented 
to specific goals and needs of NIE programs. Evaluation criteria, 
dissemination plans, deliverable products, must all be specified, 
though with the right to redirect them as the years pass. 

Our recommendations include both the general goals just outlined, 
as well as some sketches of a process for moving towards them which 
we think should begin immediately. However, the need for a9tion on 
many present lab and center contracts, and for spending Congression- 
ally earmarked FY 1976 funds, will no doubt make it impossible to 
conduct many institutional reviews in time to integrate decisions 
on past contracts, current FY76 one-year awards, and future special 
state's. We recognize that some decisions must be made on the quality 
and relevance of specific projects, without such an overall context 
to place them in, and we accept that though we deplore the circum- 
stances and history which make it inevitable. 
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We believe that institutions in danger of losing substantial 
funds as a result of such judgments should be candidates for the 
earliest review, not so much on the assumption that high quality 
has been overlooked completely, but to assure that no re-configu- 
ration or change would alter the project-by-project decisions 
made. For the rest, we would expect that criteria and procedures 
would be decided and a system in place early in 1976 so that re- 
views could proceed, with the entire set examined closely within 
three years from now. (Note that we do not set a target of a def- 
inite number of institutions which must be in place in the new 
relationship by the end of the process. We expect no more than 
six to eight will qualify, but it may be less.) Extensive dis- 
cussions with the profession, the public, and Congress may be 
useful along the way, to explain the goals and the process being 
followed. 

And, in keeping with this schedule, we would expect the 1977 
NIE planning process to include attention by the Director to pro- 
viding the necessary staff and resources for such a review. And 
competitions for laboratories and centers in Fiscal 1976 could 
well forecast the future relationships we have in mind, by stating 
in advance specific NIE priorities in considerable detail, rather 
than encouraging submission of various plans and hopes that may have 
little relevance to NIE areas ^ In this way FY 1976 funds would be 
used deliberately to emphasize a transition of policy, rather than 
simply spent on hastily-designed continuations or expansions unre- 
lated to likely future decisions. 

Thus, we return to our notion that NIE work towards a small-- 
er set of high quality institutions with which it will work very 
closely to carry out its missions, managing them towards goals 
the agency and the institutions can comfortably share. In return, 
the performers receive firm assurances of long-term support. Eval- 
uation during and at the end of the time-limited charter would de- 
termine whether the relationship should be renewed and on what 
terms. This result will not come about from piecemeal competitions 
and RFPs. It must be planned for deliberately, and the special status 
capitalized on for the advantages it brings for both sides, not seen 
as some sort of continuing embarrassment to be voided as soon as pos- 
sible. Monthly meetings of directors; program, financial, and man- 
agement reviews every so often; perhaps an Associate Director for 
Special Centers; these are some of the management tools that may be 
needed. The environment now is not that of 1972, with its heady 
hopes for rapid expansion; NIE policy must reflect that change and 
must seek to build on what exists in the most careful way possible, 
not leaving to chance the major decisions about the future of a signi- 
ficant R&D resource. 
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A new role; providing leadership to the R&D system 

We recommend that the NIE, through its Director, Council, 
and a new staff unit, take on vigorous analysis and leadership tasks 
with respect to the overall education R&D effort in the nation. 

From our reading, visiting, and corresponding, we were re- 
peatedly struck by the diversity of what is happening in the field, 
and the need for NIE to know about it, analyze it, and speak out 
for the role of E&D • It was the initial intention, and we think a 
very good one, that NIE be the agency of the Federal government not 
only for supporting programs, but also for developing policy with 
respect to educational research and development. NIE does not, and 
need not, have in its budget all the Federal government's dollars 
spent in support of education R&D, But it should have enough of the 
total amount to be a balancer, to bring about coordination and in- 
tegration among the Federal programs • Sufficient amounts should be 
allocated to the various' mission agencies because they can do a bet- 
ter job of meeting specific agency objectives than NIE is able to 
do as a general-purpose agency. Also, allocating funds broadly can 
promote desirable interagency competition. 

However, NIE seems at present to give practically no attention 
to the planning and coordination of the entire national effort in 
education R&D. Of course there cannot be a tightly-controlled, top- 
down system of policy planning run by NIE. But NIE should have suf- 
ficient staff at a sufficiently high level that it can develop both 
an informational and an analytic overview and understanding of what 
is going on in the Federal government and outside, what the key issues 
and accomplishments are, what the key problems and failures are. Its 
rgle in these respects should be intellectual and its approach should 
be to lead by force of analysis and intelligence rather than to lead 
by dictating. Its influence over policy in the Federal government 
would come primarily from the depths of its knowledge and the imagi-- 
nation of its proposals rather than from bureaucratic power. Its 
impact on research elsewhere will similarly depend on the quality 
of data, insight, and analysis, but also on aggressive publications, 
and professional contacts by credible members of NIE staff. 

All this will require a very sophisticated research and analysis 
unit at the very top of NIE. It may also call for broader authority 
than NIE now has, to obtain access to information and developments 
going on in other Federal agency R&D programs and outside as well. 

The present handling of several potential methods for this role 
leaves much to be desired. First, a small unit with the responsibility 
to study and monitor the R&D system is lodged in the Dissemination of- 
fice, with useful plans but limited support and apparently little re- 
lation to broader policy. Second, the two statutorily-required annual 
reports, one to the Assistant Secretary of HEW for Education on the 
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current status and needs of education research, and another to the 
President and Congress on education and education research in gen- 
eral, would seem prime opportunities for displaying such analyses 
and speaking out for how the whole might be more than its parts. 
Rather, these reports seem to be taken as onerous chores, and the 
first report to the President only recently limped from the press 
in the form of a tedious and ill-explained list of contracts and 
grants. 

Now that several uncertainties are behind the Institute and a 
permanent Director is in place, «uch a new role can seem more ap- 
pealing than it would have been under a "zero budget." A good 
deal of excellent work exists both under NIE sponsorship and else- 
where than can be built upon, and we note again the conceptual frame- 
work presented by Sam Sieber in his appended paper, thus it is not 
as though the analysis would need to start absolutely from scratch 
or without guidance. So long as the nation's R&D capacity is as 
limited as we think it is, every bit of intelligence used in think- 
ing through how to use it best will be well-spent. In the political 
arena, too, a well-thought-out position commands respect, even if 
not agreement. NIE can lead the Congress much more forcefully than 
it has, with strong data on gaps and needs in educational research 
and development. 
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VI CONCLUSIONS AND 
RECOMMENDATIONS 



We end by sunnnarizing our conclusions and being specific about our 
reconnnendations. Our conclusions are placed in three categories — 
concerning the R&D system, the NIE, and the regional laboratories and 
R&D centers. The recommendations are addressed to Congress and the NIE. 
As a context for these thoughts and suggestions, we want to comment in 
general about the whole enterprise before moving to specifics about 
where it is today and how the Federal role might be strengthened. 

To understand the effort that the United States is making through 
WiE to support educational research and development that will produce 
demonstrated improvements in teaching and learning, one needs a world 
perspective. Looking at the economically well-developed countries of 
the world, one finds precious little successful, applied educational 
R&D. There are some centers in Sweden, the United Kingdom, Germany, 
Israel, and a few other places where disciplined inquiries about 
education by social and behavioral scientists are providing the basis 
for changes in educational policies and program. But the general 
picture is one in which expectations exceed performance, and measurable 
improvements in the learning and behavior or students are difficult to 
demonstrate. 

In the less-developed countries educational research and develop- 
ment has even further limitations. The supply of trained social and 
behavioral scientists available to work on R&D is so limited that the 
first task in many countries is to prepare capable people. Work now 
under way tends to focus upon the evalixation of a few experimental 
projects or upon accurately describing some of the problems that exist 
prior to attempting their solution. 

Seen in this perspective the United States has a rather large and 
-sophisticated commitment to educational research and development. 
Indeed, many nations of the world look to us for guidance and stimula- 
tion in the field. Yet if this is true, why do we find ourselves 
discontented or even disenchanted with the results of what we are about? 
The answer to this query is not simple, but it must certainly include 
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the following: 1) Social science research generally is only slowly 
developing the sophistication that allows it the luxury of predictable 
results; 2) The problems of bringing about and measuring changes in 
human learning and behavior are vastly more complex than those of 
technological change and are cut across by difficulties of cultural 
tradition, linguistic style, and emotional factors that simply do not 
exist to the same degree when one is dealing with things rather than 
people; and 3) The need for improvement in the results of education 
is so clear and so great that all interested parties (legislators, 
educational policy makers, teachers, and parents) develop an initial 
enthusiasm for educational R&D only to have its halting and limited 
results seem at the least unsatisfactory and at worst inexcusable. 

Against this background it is well to ask whether investments 
in this difficult field are worth the money. The only possible reply 
is that we must keep plugging away at the difficult problems of learning 
and teaching and that doing so by orderly scientific inquiry is almost 
certainly better than by hunch. More is known today because of educa- 
tional research about how to motivate children, about how to develop 
and try out methods and materials, and about how to measure outcomes. 
That we do not have the final answers in any of these realms is not a 
valid reason for abandoning the effort. For the United States, NIE is 
the central expression of that difficult and frequently frustrating 
enterprise. 

Conclusions regarding the R&D system 

1. The set of people available to do education R&D and link its 

results to schools is too small for the job, at most no more than 10,000 
people altogether, and fewer than that in person-years of effort. 
Regional laboratories and R&D centers include over ten per cent 
of the total staff now doing R&D, and consequently must be viewed 
as a substantial resource. 

2. The R&D system outside the labs and centers is fragmented and 
hard to organize owing in part fo the meager number of R&D 
groups in universities, consulting firms, state education 
agencies, local school systems, and other places. This 
fragmentation lays a heavy burden on NIE to reach out in 
creative ways to bring the scarce and dissociated resources 
together to work effectively. 

3. While basic research must continue, and needs support wherever 
it can be found, an even more pressing need is for both concrete 
program development and readily available aid to local schools 
in using the results of program development done elsewhere. 
Merging the functions of research, product and program develop- 
ment, and service to schools in institutions such as labs and 
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centers may cause inefficient use of available energies, anct we 
stress clarification and even possibly a more specialized set 
of functions at least for some national laboratories. 

4. Though we consider there to be a need to protect both university 
researchers and the renewed national laboratories from undue 
pressure to serve schools directly, there remains then a gap of 
technical assistance and support for schools which are trying 

to improve. Current approaches conceptualize the issue either 
in mechanical terms of how to "disseminate" from a national 
R&D bank, or exclusively in terms of local self-help. The 
former notion is full of substance (products, research findings, 
teacher guides, exemplary practices) but lacks any motivational 
basis. The latter is content-free and imbued with social- 
psychological assumptions about the nature of innovation and 
self-renewal . Our conclusion is that neither strategy alone 
will make much difference to education, and that they must be 
integrated. We are discouraged at the predominance of discussion 
in much of the field and in key parts of the Executive Branch 
outside NIE simply in terms. of the mechanical model of 
dissemination. 

5. Each of the system's institutions has its own sense of function 
and interests, and with few exceptions, inquiry into mission pri- 
orities is not one of them, nor is deliberate change in response 
to the results of inquiry — one's own or someone else's. It is 
thus extraordinarily difficult to involve people in an R&D agenda- 
setting process in the first place, and harder still to attain 
credibility for the more searching questions that might be asked 
about present educational arrangements. NIE should receive credit 
for attempting to build some consensus around certain areas of 
focus for its R&D work» 

6. A variety of persons and divisions in universities, and in state 
and local education agencies, are underused or even ignored as 
parts of the national R&D effort. Minority persons and women 
are especially underused in the overall R&D effort. 

7. We may be seeing a trend in the world of education R&D towards 
emphasis on marketing, the fierce rush to attract new business 
for R&D performers, and diminution of emphasis on perfoinnance — 
in research, product quality, or attention to ultimate needs of 
schools. This legacy of shrinking funds — the resulting scramble 
by R&D institutions to maintain themselves — is not likely to 
change in the next few years, but will have pernicious consequences 
if it does persist. 
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Conclusions regarding the NIE 

8. The staff we have met seem able, perhaps above the average 
caliber of government staff, and they seem to be working 
remaikably productively in face of many difficult circumstances. 
We conclude that despite recent Civil Service criticisms and 
resulting uncertainties about future personnel policies, the 
Director and Council will need vigorously to defend NIE's 
partial exemption from civil service recruitment and hiring 
practices. Many more top-level research managers and experi- 
enced practitioners will be needed in the next few years, 
and forced hiring from within government ranks will make that 
very much more difficult. 

9* Staff appear to have diverse perceptions of the major role or 
objectives of their agency. By this diversity, we mean that 
program staff appear almost without guidance as to how to relate 
to individual pieces of work under way through grant and 
contract, how deeply to be familiar with them, how much to call 
on others for help in monitoring particular aspects, how much 
to intervene, how much to be an advocate or critic. We feel 
such diversity has caused great difficulty for NIE in the field 
owing to inconsistencies as project officers change. 

10. The haphazard policy of monitoring, and the dictum that all 
new funds be thrown to opai competition, seem likely to combine 
to reduce the incentive for performance of quality v/ork by 
contractors and grantees, since outstanding performance is 
recognized only in hit or miss fashion. We note that in 
several othetr R&D agencies, continuation grants form a 
substantial fraction of the awards, and review of such 
continuations provides at least some quality control. (Though 
considerable funds were given out in "continuation" awards in 
recent years, they were in fact incremental funding decisions, 
not true reviews and fresh decisions to continue a line of work 
building on the past.) 

11. Diversity of approach to monitoring and review by project 
officers result in part from the past NIE policy of ignoring 
labs and centers as coherent entities, and relating only to 
projects. VJe find such a policy confining and insupportable 
in the present context and for the near term. It results also 
from almost total lack of agency-wide attention to issues of 
evaluation and quality standards which we find hard to justify. 
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12. While there appear to be excellent relations in many cases between 
project officers and projects, based on idiosyncratic mutual 
adjustments in each case, there Is strong distrust and miscommuni- 
cation between the field and NIE at higher levels, since decisions 
such as general funding for types of activity appear to be made with 
little or no knowledge about institutions or projects. 

13. We are disappointed not only at the inconsistent amounts of 
information available — ^^and the lack of policy governing such 
knowledge — at the project and institutional level, but at the 
level of components of the R&D system as well* We conclude 
NIE has not yet taken on its rightful role as intellectual 
leader of education R&D, both within and outside the government 
setting forth directions and analyses on the subject based on 
wide knowledge and top-quality analysis. 

14. For giving direction to its own funding and management, we 
conclude that the new NIE program structure centered on problems 
is prudent in the short-term, though we hope that in time both 
basic studies and internal research can gain prominence along 
with extramural work directed towards solving specific problems. 

15. The process of choosing sub-priorities and strategies for 
attaining the goals of each problem area does not reflect as 
much care for broad participation and building legitimacy, 
nor for taking into account the work presently under way, 

as did the initial choice of problem areas, at least in the 
eyes of the field. 

16. We conclude that poor communication with the public, and potential 
performers of NIE's work renders less meaningful the idea that 
open competition is a useful strategy for finding the best possible 
performance. Little imagination seems to havc; gone into linking 
NIE R&D needs with potential performers through creative adaptations 
of the typical procurement nethods used for nuts and bolts. 

17. In particular, little attention has been paid to bringing the 
local schools and state agencies into partnership with NIE as 
agents of R&D, not simply passive recipients at the last stage 
of an R&D process. Attempts to improve states' ability to 
disseminate products and knowledge produced elsewhere are an 
important effort, but to some degree perpetuate notions of undue 
functional distinctions among agencies and a mechanical dissemina- 
tion process that we feel do not represent reality, and should not 
be enshrined in NIE policy. We do not mean to imply a conclusion 
that no help be given to states to develop an extension or 
dissemination system, only that state and local agencies' capacities 
for engaging in other functions of R&f) need to be equally developed. 
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18. Whilei we cannot claim to have made a thorough review of the 
relationship between NIE and the Department of HEW, signals 

from a number of sources suggest that there is room for improvement. 
The considerable emphasis placed on making NIE independent of the 
Office of Education may have had the effect of giving it an 
unintended distance from the Secretary's office in HEW. There are 
indications that NIE has not received supportive leadership in 
formulation of its policies and in gaining Congressional acceptance 
of its ^budgets and appropriations. While NIE is a small agency in 
HEW in budgetary terms, it has a powerful potemtial for the educa- 
tion system of the entire nation — an enterprise on which close to 
$100 billion is expended annually r Ws couclude, therefore, that 
both NIE and the Secretary of HEW need to give further attention 
to developing a relationship that will advance the mission of NIE. 

19. Despite the continuing controversy about the value of curriculum 
reform as a strategy of change in education, we see little clear 
policy analysis at NIE on the subject. 

20. We understand the political pressure for "dissemination" of the 
results of R&D, but we conclude that NIE has done little to attack 
the problem as a substantive matter or cluster of issues and 
competing conceptualizations. We do not think that work in the 
field can be halted until theory catches up, but we do believe 

an experimental attitude would be helpful even as action goes 
forward, and that diverse groups within NIE could be brought 
together more directly to consider paradigms for change and the 
various roles of "dissemination" within them. Research on know- 
ledge-utilization could be more extensively funded as an essential 
basis for policy in this area. 

Conclusions regarding the regional laboratories and R&D centers 



21. Failure of the laboratories to reach some goals held for them at 
the outset seems to us chiefly a failure of the government to guide 
and encourage them towards those goals, not a failure of the concept. 
The concept of a specialized, separate agency in touch with schools 
but able to retreat from direct service to test ideas and develop 
new programs still seems distinctive and sound and worthy of 
extensive support. 

22. Actual performance by tha laboratories and R&D centers appears 
to be widely varied by any standard, and judgment is complicated 
by the variety of activities within and across institutions, as 
well as by the complex set of claims and counter-claims about what 
each lab or center has in the past been directed to do. Regional 
service, contributions to knowledge, proven product impact or wide 
dissemination, all compete for preeminence in various eyes. We are 
concerned lest hasty review of product impact, for Instance, be 
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used as the exclusive measure of a laboratory's present value or 
future capcity. It was not possible for the consultant group to 
develop and apply a detailed set of criteria, and we believe the 
NIE staff should do that work in any case. Considerable work 
will need to be done, judging from our discussions with staff, 
to arrive at criteria of quality and relevance that can be 
shared • 

23. Despite this difficulty of assessment, we find at least strong 
rhetorical agreement among this set of institutions that they 
do not expect "institutional support" unrelated to performance 
and to the relevance of their work and that they would welcome 
straightforward and tough-minded oversight by consistent and 
qualified government officials and outsiders. 

24. We conclude that the persisting dilemma for Federal policy towards 
the laboratories and centers has been the large number of institu- 
tions to which the government seemed committed, and their very 
mixed quality of work. Since the first few years of each program, 
efforts have been made to both reduce the numbers and encourage 
higher quality. We conclude that there may have been more success 
in eliminating marginal institutions, or at least ending their 
substantial Federal support, than success in improving the quality 
of work at those remaining. We find it essential that both trends 
of policy continue, to result in a still smaller number of institu- 
tions dependent for major support on the NIE, with improved quality 
and relevance of effort, and related to the NIE in special and novel 
ways . 

25. Moving from the present situation to the one we envision — of more 
diverse ties to universities, and a smaller set of national 
educational R&D institutions — will not be accomplished effectively 
by allowing individual project contracts to lapse and beginning 
new work through piecemeal sole-source awards or open competitions 
in which labs and centers are simply free to enter along with 
others. Instead, we conclude that NIE must give priority in 
planning and procurement to dealing holistically with each of the 
present set of institutions, with the goal of preserving and 
strengthening a smaller number with significant potential. 

26. We expect that no more than six to eight of the present laboratories 
and centers would meet comprehensive criteria of quality and 
relevance to NIE's missions, upon close examination of entire 
institutions. (Other estimates we have received are even lower.) 

We do not resolve upon any particular target figure for the number 
of institutions that should emerge from such a review. Those 
which do meet criteria of overall quality and relevance will need 
to be substantially funded to provide a necessary "critical mass" 
of R&P activity. 
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27. After a deliberate process of review of all aspects of each 
present lab or center, and consideration of redirection or 
reorientation in each case, it may then be appropriate to 
consider new candidates for a special relationship with the 
NIE. We conclude that extensive new competitions seeking 
such candidates are unwise, and that bringing existing 
non-lab-or-center organizations into such a special status, 
or encouraging the establishment of new places, must be done 
with great care. 

28. We conclude that in any case, no more than about a third of 
NIE's program funds be allocated to work at the resulting 
special institutions. 

R ecommendations for the Congress 

1. That NIE be reauthorized on a long term or permanent, and 
with authorization of appropriations substantially above 
the $80 million per year in the administration bill. 

2. That explicit authority be given the NIE, if necessary, to allow 
it to collect data from other Federal agencies concerning 
their education research and development activities, and from 
organizations outside the Federal government as well. 

3. That the new legislation on the Institute continue the concept 
of a strong policy-making National Council, including terms 
longer than the present three years. 

4. That the new legislation continue the authority for the 
Institute to staff its activities flexibly, to allow for 
bringing in the uniquely talented individuals that are needed. 

Recommendations for NIE 

5. That the present problem-foci be retained but that these be 
strengthened by establishing a formal advisory body for each 
program which could link the Institute with various constituen- 
cies for its work in each area, aid in reviewing specific 
awards thus freeing staff to consult with proposers, and serve 
as a sounding board to insure that the "definition of the 
problem" remains current. 

6. That much detailed thought be given to the role of project 
monitor and how NIE wishes monitoring to be performed. Such task 
force thinking should be followed by appropriate written policies 
and staff training. In the course of developing such approaches 
we urge the study pf practice in other R&D agencies and 
consultation and surveying of present grantees and contractors. 
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7. That the Institute establish an ''R&D System Monitoring and 
Analysis Unit" at the highest level, which can have staff and 
resources to take active part in the Director's decision-making 
and planning, and in the required reports to the Department, 
the President, Congress, and the public. The 1975 Databook is 
an important first step and should be built upon by the new 
unit. We would hope that the technical experience of gathering 
data for the volume might be the basis for a national meeting 
on sources of data on education R&D and how to strengthen the 
collection and reporting of necessary indicators, with advice 
sought both from the field and from other agencies such as 

NSF with wide system-monitoring responsibilities and experience. 

8. That NIE completely review and revise its efforts to inform 
the R&D system about NIE. We urge the mandatory publication 
by program of f ices of their plans and procurement expectations, 
and names of people who can be contacted for further specific 
information. If mailing lists continue to pose problems for 
those who do wish to communicate with various groups we hope 
outside help will be, sought to devise a modem and effective 
system. We make other recommendations concerning publications 
under the recommendations on procurement and state and local 
agencies below. 

9. That the NIE devote explicit and public attention to issues of 
the present value of curriculum development as a strategy for 
aiding schools. We sense strongly - held and diverse views on 
the subject, and there appear to be policies made, in part, 

on private judgments of the merits of the case. But we notice 
little public airing of the debate. Some writing, convening, 
and general discussion of the value of Federal support of 
curriculum development would be a national service, as well as 
useful in reaching and explaining a key NIE policy choice. 

10. That since the goal of quality work requires a diverse array of 
procurement approaches suited to each type of R&D work, the state 
of the community of performers, and the preferred role of the 

NIE staff, NIE should devote resources in the near future 
to strengthening its repertoire of available tools and the 
abilities of program managers to use them. We recommend, for 
example : 

° Review of the contracts and grants office staff size, and 
the adequacy of their backgrounds, for insuring timely 
and relevant advice to R&D program managers on procurement 
and business management techniques suited to the special 
circumstances of R&D. 
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Closer working relations between contracts staff and 
program staff, from the inception of design of a procurement 
so that plans can take into account the substance of what is 
wanted, how to reach those who should know of it, and a 
range of possible ways to conduct the procurement and award 
the funds. More formal circulation of "good" announcements 
and RFPs, witn key points highlighted by contracts and 
other staff, would be educational, also. 

Review of R&D procurement procedures used elsewhere in the 
Department, as well as in AEC, NASA, and NSF to gain a 
broader sense of options available. Among the possibilities 
that should be explored are: 

— greater use of restricted competitions, such as by 
region or qualification; 

— use of review processes which allow program staff 
to work with proposers at an early stage; 

— based on identification of resources at a university, 
use of block grants to a dean or other individual with 
discretion as to how to fund and organize the resources; 

— use of continuous competitions in areas that have 
continuing interest to the NIE, with open dates and 
deadlines publically announced long in advance and 
awards made periodically; 

— funded design competitions. 

Exploration in the near future of the potential of several 
new publications relating to procurement: 

— a general guide to obtaining NIE funds; 

— explanations and instructions covering all parts of 
grant and contract competitions, including certifications 
competitive range, clarification question phase, 
negotiations^ and award, including notice of Institute 
commitment to debriefing; 

— monthly or other regular listing of competitions in 
process and contemplated. 

Establishment of a procedure for recording judgments of 
past work by grantees and contractors, and procedures for 
using that information fairly in future competitions or 
other procurement actions. 
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Exploration of the possibility of regional seminars, such as 
those held by NSF*s RAM (Research Applied to National Needs) 
Program, to acquaint people with research directions and 
application details, for a number of NIE programs at once. 

Complex questions of law and policy arise in some of these 
suggestions, and we imagine a substantial study effort will be 
needed over the next year, perhaps including several invitational 
conferences of people with wide familiarity with R&D management 
to consider the state of the art of R&D procurement in education 
and its impact on the R&D system. Continued attention will be 
needed after development of better tools of procurement, to 
insure that program staff are aided in using them fully. Few 
other tasks are more important, 

11, In program planning and in procurement, special attention must 
be paid to overcoming the assumption that schools, school 
districts, and state agencies are merely the recipients of 
the fruits of R&D, We suggest as one possibility that NIE 
establish a state-and-local-agency liaison unit, designed to 
monitor R&D being done by these organizations, to link them 
together in useful way-^;,. and to draw their attention to NIE 
activities of special interest. All procurements should be 
monitored by this unit, to avoid embarrassments such as in 
the past when states or local agencies were inadvertently 
left off eligible lists. The unit could perhaps be part of the 
R&D System Monitoring and Analysis office we recommended 
above. In addition, the unit could help these agencies apply 
or compete when appropriate. We hope NIE will consider a 
special publication for local school districts ai7i state 
agencies, designed to make them more aware of R&L enerally 
and. the NIE role as well. Contracts with ot|ier organizations 
should have clauses which encourage them to be sensitive to 
the concerns of teachers and administrators in all phases of 
their work or which could even go so far as to require " - ^ > 
participation of local schools' staffs at key points in R&D, 

Though we agree with the overall direction towards problem- 
focus, we urge the Director to reserve a modest sum for 
special opportunities which fall outside the problem structure, 
and which may suggest new problems needing attention. (We 
comment below on the need for a general long-range planning 
and problem-sensing process.) 
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That In connection with the R&D System Monitoring and Analysis 
unit we recoiranend, the Director consider helping the Coimcil 
establish a special advisory group to itself on the R&D 
system. This group of experts on science and R&D could 
review the "map" generated by the staff, and comment on it 
for the Council in reports from time to time. Such a double 
input would give the Council both expert staff work and 
analysis, and also some reflection on its meaning for policy 
by a small group which could build up substantial insight if 
allowed and encouraged to persist over several years. 

Because we are concerned not to lose the tie between education- 
related research and the larger scholarly conversation, we urge 
that NIE make special efforts to inform diverse elements of the 
university and college faculties of its programs and opportunities. 
That NIE find ways in procurements to allow the vital brokering 
of diverse interests that would not come together without an 
outside vision. That NIE maintain contact with such general 
scholarly associations as the Social Science Research Coimcil, 
American Council of Learned Societies, and American Association 
for the Advancement of Science. One suggestion that we reiterate 
in this connection, is the idea of awarding block grants to deans 
or other university officers to support work by younger faculty, 
or to carry out a specific set of activities. 

(a) Because of our conclusion about the under-supply of 

education R&D personnel, we have given some thought to 
methods for increasing it. We do not have a plan to 
recommend, and we recognize that both tight budgets 
administration policy argue against some of our ideas. 
But we commend these scattered notions to the Director 
and Council for further study. We think that even 5% 
of the NIE budget could be a very useful investment 
at this point, if spent in some of the following ways: 

Awards and incentives to individuals from a 
variety of disciplines, to pursue graduate 
study leading to a doctorate related to 
education research; students to attend wherever 
they wish, and to finance part of the cost 
themselves ; 

. To convert recent graduates in other fields, 
a program of post- doctoral fellowships in 
education research; 

• Mid-career awards to allow mature investigators 
to ease into a new field without loss of job; 
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Apprenticeship opportunities to be required in the 
terms and conditions of all NIE grants and contracts; 

Awards for graduate study for staff in state and local 
education agencies, on condition they plan to 
return to their home base for a period after receiving 
the degree in a research-related area. 

(b) Because of our conclusion concerning the under-use of 
minority persons and women in the national R&D effort, 
we recommend that NIE strengthen its policies in support 
of increasing the participation of these groups in R&D. 
Such policies could include: 

Special reviews of each procurement to note 
opportunities for minority firms, and better 
maintenance of lists of qualified firms for 
use of programs; 

Aggressive review of hiring by contractors to 
insure compliance with equal opportunity and 
affirmative action law and regulation; 

Special attention to expanding training and 
apprenticeship opportunities for minorities 
and women in all NIE grants and contracts. 

16. Because we sense that attracting the very best staff may depend 
on offering opportunities for research as well as for monitoring 
the work of others and because research management requires famil- 
iarity with the state of the art in the field, we recommend a 
modest planning effort to inquire how intramural research activity 
can be made a part of the normal expectation for NIE staff. We 
recognize that serious in-house research must await higher fund- 
ing levels, which will only come as extramural work shows credible 
results and growing constituencies. However, we strongly urge 
retention of the NIE Fellows Program as a start, and some for- 
ward planning about the shape of an eventual internal research 
activity. 

17. Again we make a recommendation which must await higher funding, 
but which we feel strongly about. Most research in higher 
education is done elsewhere than NIE, but even in a limited 
way NIE could make a contribution now as a prelude to later, 
more extensive efforts. We have in mind the area of higher 
education finance as a start. We would hope that eventually 
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NIE might supF^-t R&D pertaining to a variety of post- 
secondary institutions, including those outside formal 
schools such as museums, parks, zoos, libraries, and learning 
networks • 

Though we understand the need for more short-term-payoff work 
just now, we recommend that NIE program areas not reject 
longer, more complex "development" activity without careful 
analysis. We have in mind a number of neglected areas of 
the education program of the country, any one of which could 
profit from some clear thinking and program development 
and trial. NIE needs, and the needs of schools and teachers 
may in part conflict here, and we would be disturbed to see 
too easily written off the whole idea of development. 

That the NlE establish an internal task force at the very least, 
if not an ongoing external study and advisory group, to attack 
the substance of the term "dissemination." We sense diverse 
conceptualizations within the NIE and insufficient effort to 
reach synthesis, or even clarification of complementary features 
or differences. Thinking in other quarters of the R&D system 
on the subject is not well-advanced either, and could be immensely 
stimulated and enlarged by some NIE writing and convening on the 
subject. We urge that internal planning at NIE for its own 
program consider how the "dissemination" and "problem-solving 
approaches will merge or be integrated at the operating level 
of the school, so that both materials and ideas, as well as the 
motivation to use them come together for sustained improvement. 
We have noted the strong potential of "extension agents" 
for such a role, and we urge that efforts to promote a major 
program of support in each state be augmented. In particular, 
the possibility of pursuing a "top-down" strategy in creating 
a national extension system should not be ruled out. The 
present "bottom-up" strategy (of building on what exists) is 
useful, but might result in a weak system without common • 
standards or wide visibility and support. 

That NIE establish a long-range planning process (in addition 
to the process for choosing the next increments of work within 
the present problem areas). We recognize that NIE is not 
completely it own master, and that of course events will press 
new issues upon the Institute regardless of the best prior 
planning. But we see no present deliberate activity aimed at 
sensing the state of education generally to maintain a timely 
list of candidates for NIE attention—nothing other than the 
atomistic and informal actions of the Director, the staff, and 
the Council. Some close work among the Assistant Secretary s 
office, the National Center for Education Statistics, and 
Congressional committees would be a start, with perhaps joint 
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collaboration on some indicators that would serve to alert legis 
lators, the USOE, and NIE to needs to be addressed through their 
respective methods* 

The following series of related points constitute our thinking 
about a course of action towards regional educational labora- 
tories and R&D centers. 

(a) We recommend that NIE adopt a long-range goal of 
assuming the majority support of a small number 
of large, high-^iuality R&D organizations, with 
whom it will have a relationship different from 
that with other grantees and contractors. 

(b) The national education R&D institutions which are 
the goal of the above recommendation should share 
certain features: 

,° Emphasis on a single mission, closely tied 
to one of NIE's national R&D priority areas; 

° A purpose of following ideas from inception 
to utilization, with specific tasks along 
the way firmly agreed to in advance; 

° Stable funding for three to five years, at 
a level of at least $3 to $4 million per year; 

° Funding chiefly from a single source — the NIE, 
with other funds subject to review and possible 
limitation to maintain essential mission-focus; 

° Protection from demands to give local services 
unrelated to field activity that is part of the 
R&D mission; 

° Close ties to the major sponsor — ^NIE, for review 
of the entire institution at intervals during and 
at the close of the contract term, including review 
of finances and management, as well as program. 

(c) We recommend, therefore, that NIE review and revise all 
present policies that contribute to the present situation, 
where it is substantially supporting a relatively large 
number of special Institutions, of diverse quality, with 
varying lengths of contract term, subject to uncoordinated 
NIE management and review, and inconsistently related to 
NIE priorities. 
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We recommend that present regional laboratories and R&D 
centers be considered candidates for this new relationship 
to the national R&D effort and redefined mission and 
obligations. 

We recommend that all procedures for dealing with the 
laboratories and centers for the next few months be 
immediately reviewed to insure that actions now under 
way are not in conflict with the comprehensive policy to 
be outlined below. We recognize there may be cases where 
the institutional review procedure to be spelled out 
cannot be completely followed, and action must proceed* 
But we have in mind particularly not following through 
on any planned competitions or other funding actions 
for labs and centers that do not relate directly to NIE 
problem areas, and which do not lead towards a definitive 
policy to be followed in future years. 

After estimating the available funds for the special 
institutions according to guidelines of how much in 
total and how much each, NIE staff should begin a 
systematic review of each lab and center according to 
a number of criteria: 

Relevance of work to NIE priority areas ^ 
Quality of work 

Quality of management, both programmatic and financial 
Commitment to utilization of results 
Perceptiveness of how to keep big R&D ^targeted to 
help schools 

Willingness to accept the obligations of the new 
institutional form being proposed 

As we have stated, considerable work must be done to 
refine such a list, make it operational, and set standards 
in each area. The reviews should take into account 
potential to contribute, not simply past work alone, as we 
strongly favor the redirection of capacity before its 
abandonment. Based on the available funds, up to the 
target ceiling, some of the highest quality and most 
relevant institutions should be supported under the 
conditions we will specify; others would not, but would of 
course be eligible to compete in other NIE activties. 

An independent external patJ^l of R&D practitioners, school 
and university leaders, and members of the public, should be 
convened to examine the institutional review process and the 
resulting judgments and proposed actions, and report their 
conclusions to the Director and the council. 
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The institutions selected would receive the type of mission- 
specific contract for three to five years that we have 
discussed. Extensive and complex work in a particular NIE 
problem area would be expected, with timelines, program 
management, and expected results so far as possible 
specified at the start. The support would not be 
institutional, in the sense that the organization could 
decide how it would be spent. Beyond the development of 
the initial detailed contract, NIE would reserve the right 
to intervene strongly in program and management if work 
falters or priorities change. In return for extraordinary 
continuity and magnitude of funding, the special institution 
would agree to these close ties with its sponsoring agency, 
and to reexamination of the role of its policy board. 

Applying the above procedure immediately to all labs and 
centers is not feasible b ecause of the limited time 
before most lab/center contracts expire. Thus, only a 
small number of institutions can receive immediately the 
detailed review we recomnieud. The criterion for choosing 
places to include in the first set for comprehensive review, 
is the NIE staff estimate that they are unlikely to be 
successful in finding other funds at NIE, on grounds of 
quality and/or relevance. Other places with more substantial 
prospects can safely be postponed, but reviews must be done 
soon of institutions where piecemeal decision processes seem 
likely to result in serious loss of Federal support. 

However, immediately following completion of reviews of 
institutions in jeopardy, NIE should begin reviews of all 
laboratories and centers in the same fashion as outlined in 
(f) above. These reviews could be spread over several years. 

We recommend that NIE not hesitate to end any sense of 
general obligation to labs or centers which, upon thorough 
institutional and programmatic review, seem not to have the 
quality of work or relevance of work that NIE needs, or the 
likelihood of attaining it upon redirection. No institution 
should be considered for inclusion in the special category 
we have been describing simply on the basis of its own track 
record and desire to continue doing the same. Such a 
special relation is reserved for highly focussed work of 
direct relevance to NIE. Other funding mechanisms may be 
available to allow more self-directed work. The consequence 
of this recommendation may mean that some institutions 
actually close, or must reorient their work away from R&D 
in areas supported by NIE. We are prepared to accept that. 
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We sense that the concept of "regionality" in the original 
legislation regarding laboratories was added for political 
convenience, and we recall the contrary concept of "national" 
laboratory described by the task force which initiated the 
notion. Accordingly we recommend to NIE that some care be 
taken to maintain diversity of location of the remaining 
special institutions, so that a spectrum of potential 
problem-foci is noticed and can be placed as candidates on 
the national R&D agenda through informal interaction of NIE 
and institution staff and directors. But we urge that NIE 
avoid continuing or establishing any special institution 
simply so that all areas of the country have one nearby. 
We recognize that the closing of sizable installations will 
be a difficult task. But those places that achieve the 
special support status we have in mind should do so on the 
nationally-judged quality and relevance of their work. 

Where institutions will lose substantial Federal funds over 
a short period, we recommend transition funds be awarded to 
aid in reconsidering of goals, seeking new business, and 
reorienting staff. 

Evaluation criteria for the special institutions should be 
agreed on in advance, so far as possible, to guide interim 
and end-of contract reviews. It should of course be made 
clear at the outset that the institution has a mission- 
related life-expectancy, not an indefinite tie to the sponsor. 

Since time and careful thought are needed to design and 
implement the extensive review process we have in mind, 
we recommend that the Director review the present organi- 
zation of NIE for its adequacy to the task. Since there 
now is neither structure nor incentive that brings people 
from various program areas together to consider a particular 
institution, we think the Director should consider establish- 
ing a position to do this, such as an Associate Director for 
Special Institutions. A critical first task for this 
person would be to review the procedures, criteria, and 
result of the several past reviews of laboratories and 
centers including those by Frank Chase, Frank Westheimer, 
USOE, and the 1972 panels. This experience must be built 
upon wherever possible. 
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This paper was prepared as an addendum to "R&D Funding Policies of 
the National Institute of Education: Review and Recommendations," 
a report by ten consultants to NIE. Although a member of this group 
of consultants, the author desired to make a more personal state- 
ment than provided^^for within the framework of the report. 
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The Requirements of a National Educational R&D System 

Sam D, Sieber 
NIE Consultant 



As a group the consultants on alternative funding policies of NIE has 
appropriately centered its attention on issues of immediate concern to 
NIE and the R&D community, to the political and economic context of 
those issues, and to concrete recommendations for dealing with them- 
But lurking behind the discussion of current problems, and indeed only 
partially apprehended by the field itself, are certain assumptions about 
what constitutes a "national R&D system" in education. To my knowledge, 
these assumptions have never been spelled out, and it is the purpose of 
this addendum to express my personal view as to what these might be. 
This task, I feel, is a vital one, 

A national R&D system, we are told, is something that NIE is 
mandated by Congress to nourish and bring to fruition, something 
which the group of consultants hopes will become more viable by 
virtue of their efforts, something which nearly all of us endorse — 
and yet, no one seems inclined to define this system. Clearly, it 
is not something that exists in the natural order of things, but 
something that we would like to approximate more closely than at 
present. These considerations have led me to ask: What might be 
the design requirements of an R&D system in education? These require- 
ments, or "functional needs" to use another set of jargon, need to be 
laid out in a clear-cut fashion so that one call tell where we are and 
where we want to go — or at the very least, to stimulate debate over 
the underlying dimensions of an R&D system. Also, it might then become 
possible to communicate more effectively to Congress the fundamental 
ideals of such a system and the steps which are being tiaken to achieve 
them. 

Note that I am addressing myself to systemic' requirements and 
not to the ultimate intended outcomes of the system, such as equity, 
preparation for work, enlightened citizenship, etc. To- a large extent, 
system requirements can be ^spelled out independently of the desired 
outcomes. It is like building an atomic conversion plant: you can 
warm homes or you can make a bomb with it. The basic design require- 
ments are the same. Without a concise list of such requirements, it 
is hard to imagine how we can generate criteria of accomplishment or 
check-points for policy making and implementation, 
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Virtually the only system requirements that have been recognized 
by NIE-USOE over the past fifteen years have been functional specializa- 
tion (research, development, dissemination, training, etc.) and to a 
much lesser degree quality control . (Tbrepeat, impact on practice is 
not a system requiirement strictly speaking, but a desired outcome of a 
system. It is discussed under "goal setting.") Thus, structures like 
R&D centers, labs, ERIC, and Title IV training programs were established 
to foster each of these functions. Since other system requirements were 
not explicitly envisioned and promoted, these arrangements led to a 
variety of unanticipated consequences, some of which were quite un- 
desirable, for F„^ample; overproduction of academic researchers, meager 
attention to practitioners' interests and capacities for change, shift- 
ing emphases among functions (research, then development, and now 
dissemination), overlapping of functions within ostensibly specialized 
agencies, alienation of social scientists, and inattention to substan- 
tive goals. In short, narrow emphasis on the single design requirement 
of functional specialization has been a grave mistake. 

To be sure, functional specialization is a basic property of any 
engineered system. But the difference between a smoothly running 
engine and a miscellaneous pile of engine parts is that the former 
meets a number of requirements in addition to functional specializa- 
tion. Indeed, this particular requirement is adequately fulfilled by 
a junk yard. 

Eventually it might be possible for NIE to generate a two dimen- 
sional matrix which shows (1) design requirements, and (2) the mech- 
anisms which are intended to meet each requirement. Then, by monitor- 
ing the extent to which each mechanism is doing its job, the matrix 
might serve as a kind of score board for NIE activities. 

I will now turn to the design requlirements that I believe apply 
most critically to a national educational R&D system. Admittedly, 
these requirements need further specification, a task that NIE should 
work on continuously. 

1. Functional specialization 

I have already mentioned this requirement, and since it has become 
widely accepted and implemented, I will not dwell on it here. NIE 
should remain alert, however, to emergent needs for new specialties 
and sub-specialties and be prepared to play a role in supporting 
their development. The most obvious mechanism is training programs 
and individual fellowships; ai:?other would be "brokering" for people 
who are just beginning to specialize in a particular role, e.g., 
planners of innovative new schools, formative evaluators, local 
information officers, regional extension agents, R&D management 
consultants, etc. At present this task is not being performed, to 
my knowledge. 
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2. Balance among: 

Functions (basic research, applied research, product develop- 
ment, systems development, dissemination, technical assistance, 
training, evaluation) ; 

Performers (social scientists, professional educators, prac- 
titioners, NIE staff, other experts); 

Settings — universities, non-profits, profits; government 
labs, R&D centers, R&D teams; NIE (in~house research); 

Decision-makers and influentials — federal, state and local 
education agencies; professionals and disciplines; employers 
and businessmen; minorities; 

Supply and demand (R&D resources and the demand for these 
resources) . 

Anyone familiar with the history of educational R&D in the past 
fifty years is aware of the severe, periodic imbalances that occurred 
among these various functions, performers, settings, and participators 
in decision-making. Field service and testing were supreme for decades; 
then field-initiated research had a few halcyon years under the Cooper- 
ative Research Act; then product development was tremendously inflated; 
and now it appears that dissemination is being pushed to the fore. (In 
view of the new emphasis on dissemination, care should be taken to 
insure that we do not return to the days of field service in a vicious 
historical cycle.) Accompanying these extreme swings have' been Imbalances 
among types of performers, settings, and decision-makers, with each 
sector that was formerly privileged becoming embittered as funds began 
to flow elsewhere. By an awkward stretch of the imagination, this 
pattern might be seen as balance over time ; but do R&D needs really 
change this much from one period of time to the next? 

This point raises the critical issues of the balance between the 
supply of educational knowledge, products, systems, etc. and the 
demand for these resources. Virtually all of our R&D effort has been 
devoted to increasing supply; but what do we know about the state of 
demand? Do we assess it in any systematic fashion (a point to be 
discussed later) or do we await Congressional mandates and special- 
interest pressures to move us in different directions? And what is 
being done to stimulate demand and to make it more enlightened? Here 
is another domain of serious imbalance in the national R&D system. 

In sum, what is needed is a system which is more concurrently 
balanced with regar4 to all of the components mentioned above. 

3. Integration (or interrelation) of: 
a. Functions, 
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b. Settings, 

c. Performers, 

d. Decision-makers and inf luentials, 

e. Supply and demand. 

We know that it is foolish to assume that activities conducted in 
isolated settings by autonomous performers will contribute to the com- 
mon goals of educational reform. And yet, NIE has not effectively 
promoted relationships between research and training, development and 
dissemination, dissemination and problem-solving, labs, labs and SEA's, 
labs and universities, academics and practitioners, educational R&D 
leaders and special interest group leaders, or even researchers in the 
same field. Here is a domain where statesmanlike leadership as well 
as imaginative administration is, I think, very badly needed. Purely 
contractual arrangements can be used to further integration, e.g., 
requiring opportunities for meaningful apprenticeships in all research 
grants, requiring plans for linkages with SEA disseminators and other 
information centers in developmental contracts, requiring plans for 
coordination between programs and projects in the same problem area, 
etc. The mechanisms are numerous, but the main point is that integrat- 
ion should be focussed on as a distinct system requirement. In my 
judgment, greater integration would not only improve the system as a 
whole, but also the functioning and outcomes of each component part. 

4. Continuity in: 

a. Policies, 

b. Tasks and substantive areas, 

c. Personnel, 

d . Organizations . 

It remains to be seen whether the chronic discontinuities in 
educational R&D can be remedied without special attention to this 
system requirement. The history of R&D under USOE is rife with 
lessons. Almost every fiscal year brought a new list of priorities, 
a new proportionate allocation among programs, a new threat of being 
terminated. Turnover of personnel made it almost impossible for 
outsiders to continue a professional dialogue for more than two or 
three years or to maintain momentum for some projects which were 
originally strongly endorsed. Occasionally, after a program had been 
announced and funded, by the time an investigator could get around to 
preparing a proposal the program no longer had funds or was in the 
throes of being "phased out/' Ironically, this is the picture that 
many have of NIE, an agency which was set up to avcyid precisely these 
problems . 
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With regard to the R&D conununity at large, it is essential to 
continue structures and programs which are either too young to bear 
fruit or have already proven their worth. And even when some activi- 
ties must be terminated, every effort should be made to salvage the 
best parts by fostering new amalgams. These tactics are contrary to 
the announced policy of creating a competitive struggle for existence 
among R&D performers. In all such struggles, the powerful and the 
cunning tend to triumph, 

5, Adaptability of: 

a. Policies, 

b. Functions,. 

c. Personnel (or criteria of selection). 

This requirement might seem to be inconsistent with the need for 
continuity, and many of the discontinuities in federal educational R&D 
policy have been excused in terms of the need for adaptation to new 
circumstances. But adaptability as a system requirement must not be 
confused '-^ith bending to pressure or embracing intellectual fads and 
ideologies. Adaptations can often be made or based upon on-going 
structures and present policy. For example, labs can be reduced in 
number and can be induced to give more attention to developing '^systems*' 
rather than products, to link up with information dissemination centers, 
or to take marketing requirements into consideration during the design 
phase; ERIC can become more active in raising awareness about educa- 
tional products; R&D centers c :n be helped to do a better training job; 
and so forth. The adaptability of existing structures can also be 
enhanced by providing funds for independent research, new conceptualiza- 
tions, planning of new departures, and conferences on needed research. 

The essential point, however, is that adaptation must be based 
on continuous , systematic inquiry into the diverse needs and problems 
of education rather than on the inspirations and ideologies of R&D 
decision-makers or the pressures of stakeholders. Such inquiry should 
be informed by (1) a set of explicit design requirements for the R&D 
system; (2) a continuous study of potential and actual resources, set- 
tings and performers of educational R&D (as begun with NIE's new R&D 
Databook ) ; and (3) a continuous sensing of practitioners' needs, 
j)roblems and reactions through systematic data-collection. (With 
regard to the last point, a national "sensing network*' to identify the 
information needs of educators is now being designed under the sponsor- 
ship of NIE's dissemination division. This system could be easily 
adjusted to measure educational needs and monitor reactions to R&D 
products in general, thereby furnishing continuous, systematic data 
on the needs and problems of practitioners throughout the nation. 
The development and installation of this system deserves the Council's 
fullest support, therefore,) 
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In sum, a combination of system requirements with data on capacity . 
and data on needs should seirve as the basis for all planning, short and 
long range. Much remains to be done to realize this triadic foundation 
of planning, however. Finally, as suggested above, adaptations must be 
constrained by the requirements of balance, continuity and integration . 
These other system requirements set the boundaries of adaptability, in 
my opinion, and should not be substantially violated. 

6. Excellence or quality-control 

In the context of current pressures for dissemination, it is easy to 
forget that not all information or innovations are really worth dissem- 
inating. Obviously, if information or products are unreliable , mislead- 
ing, or unsuited to the situation, they can have repercussions which are 
altogether harmful. And even if not harmful, an accumulation of futile 
experiences with information or products might create an attitude of 
skepticism toward R&D of all kinds. Further, even when an innovation 
is workable and relatively effective, the opportunity cost of this parti- 
cular innovation rather than another might be undesirable. Although 
these points are obvious, it would seem that we are still wedded to the 
notion of "the more utilization the better." This assumption has 
prevented us from realizing that many of the so-called "barriers" and 
"resistances" to innovation and knowledge-utilization in American schools 
might actually be beneficial. 

The extent to which faddism dictates the adoption of innovations 
of doubtful merit has been demonstrated in a i-ecent study of virtually 
all big-city secondary schools. This study reveals that while 46 per 
cent of these schools are relatively high in adoptions of innovations, 
most of the innovations adopted by half of these innovative schools are 
of relatively low quality (as judged by a national panel of secondary 
school experts). Clearly, quality remains as important an issue as 
quantity of adoptions. 

There are a host of discrete mechanisms for insuring that good work 
is done and disseminated; and yet, there seems to be continual debate 
over the quality of educational R&D. The source of this paradox might 
lie in the vagueness of evaluative criteria, the lack of consensus on 
procedures, and the failure of a major sponsor — NIE to develop any 
agency-wide mechanism for assessing quality. Perhaps what is vitally 
needed is a national task force or commission on the q^uality of educa- 
tional R&D. The mandates of such a commission might include: studying 
the ways in which quality is currently assessed, the criteria best 
suited to different agencies, the qualifications of the assessors, the 
extent to which evaluation studies actually influence policy making, 
the methodology of evaluation research supported by NIE, the dangers of 
too stringent control, the area:s in which control is critical and those 
where it can be relaxed, and the dimensions of cost-benefit analysis. 
All functions (RDD&E) might be covered, or different task forces could 
concentrate on different functions. But whatever the mode of operation. 
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it would seem that an external review of NIE's quality-control proced- 
ures would give coherence and focus to this vital design requirement. 

?• Goal setting 

A national R&D system must, be able to formulate and gain consensus on 
a set of clear-cut objectives regarding output. Because these output 
goals were not clearly spelled out in the past, emphasis shifted from 
terminal to instrumental goals, or in simple language, from what the 
system was supposed to accomplish to how the system was supposed to 
operate. Thus, the functions of RD&D (plus training) became the 
salient goals. Beyond these there was the single, overarching goal 
of "improving educational practice" or simply "impact." But what is 
meant by impact — awareness, enlightenment of practitioners, adoption, 
implementation, adaptation to local conditions, planning, improved 
learning? And what are the formal aspects of impact that need to be 
kept in mind, such as scope, depth, duration, and effects? The ultimate 
goal of "impact on education" is too abstract and normative to communi- 
cate NIE's mission or to guide decision-making. 

Between this vague ultimate goal and the functions of the system 
itself there lies a vast no-man's land of intermediate objectives. 
These objectives need to be specified in operational language so that 
NIE personnel can use them for guidance and self-assessment, and so 
that practitioners, legislators and the R&D community will know what 
NIE is trying to accomplish. Further, they must be prioritized so 
that resources are net depleted and allocative decisions can be made 
with a minimum of debate. What are some of these intermediate output 
goals? Here are a few examples: 

. to gain participation of the ultimate consumer in 
development, field testing and demonstration; 

to improve the discrimination of potential adopters 
of innovations; 

' . to invent, install or nurture R&D systems in state 
and local settings; 

to improve the problem-solving anrl needs-assessment 
capacity of practitioners; 

. to nourish the development of exemplary practices by 
practitioners; 

. , to serve as a "leader" among federal agencies with 
activities related to educational R&D; 

to encourage lateral communication among schools 
engaged in similar innovations in the same region. 
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Not all goal-setting can or should be done by NIE, of course. 
Every agency that is engaged in the R&D process must be able to set 
goals that bear on the vital needs of education and that relate to 
the total R&D picture. Can NIE improve this process of pluralistic 
goal-setting without undue pressure and control? And what mechanisms 
are most useful for gaining broad participation in NIE's own goal- 
setting? 

To say that an R&D system needs goals in platitudinous. To 
conceptualize these goals in a manner which is both logical and 
precise, and to gain consensus on priorities, is a difficult task 
which, in my estimation, has barely begun. 

8. Recognition of environmental constraints and potential constituen- 
cies (to gain realism and legitimacy ) 

The diversity, autonomy, and competition among the parts, as well as 
the political constraints and reward systems of the educational R&D 
community must be taken into account in designing the components of 
an R&D system. As the consultants' report notes, educational R&D are 
secondary goals of many agencies and there is a relatively small cadre 
of full-time R&D performers, who are not necessarily the best-qualified. 
Further, public education as a profession is low in prestige; inter- 
disciplinary work is rare; huckster ism and faddism are common; there 
are numerous specialties and special interest groups with conflicting 
perspectives; practitioners resent the implication that they need to 
listen to "experts" solve their "problems"; and so forth. Perhaps the 
best terms to summarize this environment are pluralism and conservatism 
(or preoccupation with non-R&D goals). My impression is that these 
features of the environment have not been adequately weighed in the 
formulation of R&D policy and programs. There has been a naive enthu-* 
siasm for educational R&D which has not, and will never be, shared by 
the vast majority in several key groups: practicing educators, social 
scientists, activists, government officials, legislators, and lajrmen- 
This suggests two general approaches: (1) a cautious, realistic build- 
ing of individual programs with reference to existing constraints and 
strategic entry points; (2) a major attempt to gain greater visibility 
and respect for educational R&D by means of a "showcase" program with 
wide participation and high impact. 

An example of a showcase program would be a national educational 
extension network. This approach appeals very much to practitioners 
because it is oriented to their immediate needs and involved personal 
contact with someone who does not pose as an "expert." In addition, 
it raises awareness of the diversity and riches of educational R&D 
resources. It has wide coverage, as a single extension agent can 
reach approximately 300 clients per year. And it has been shown to 
work very successfully in three states in a thoroughly-evaluated 
USOE-NIE pilot program. Not only educators generally but R&D 
personnel as well would profit from an extension system if it were 
designed to serve as a massive on-going laboratory and a continuous 
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source of information about the needs and reactions of educators to R&D 
products. (And eventually the system would redound to the benefit of 
basic inquiry as appreciation rose for R&D output.) Since training 
materials and programs are available for preparing a national body of 
extension agents and information-retrieval personnel in state and 
regional centers, the time that it would take to install the program ' 
would be reduced to a matter of months. 

It is my understanding that NIE is currently trying to strengthen 
the capacity of state agencies to engage in linkage activities. But 
this "bottom up" strategy, as important as it is, might be too time 
consuming and produce a highly fragmented system which is impossible 
to monitor. In view of the potential value and timely relevance of 
a national extension system, a more aggressive "top-down" strategy 
might also be considered. 

In sum, a showcase program of this kind (similar in visibility to 
OEO's Heads tart) might alleviate many of the political and professional 
problems that confront the NIE today, and give it much greater connect- 
ion with our pluralistic educational environment. 

Concluding remarks 

I do not pretend to have covered within the brief compass of this paper 
all the design requirements of an R&D system. I do believe that the 
ones I have mentioned are the most critical, however, and sufficient 
as take-off points for further elaboration. 

Obviously, an important requirement which I have omlitted is sheer 
money; and perhaps in order to achieve the goals set forth in this paper, 
the level of funding for R&D would have to double or even triple. But 
I have cast my eyes on the future and tried to imply that the needs of 
an operating R&D system completely justify whatever resources are 
necessary to bring it about. If present funding restrictions continue, 
however, so that these systemic needs cannot be met, then I would urge 
that we disabuse ourselves of the conceit that we are creating a 
"national educational R&D system," 
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Appendix ^ 

THE UNIVERSITY OF THE STATE OF NEW YORK 

THE »TATC COUCATIOH OCPAWTMtHT 

Orricc or the Prc3idcmt or the UNtvcRStTY 
AND CoMMisaiONCR or Education 
ALBANY, NCV; YORK 12224 



URGENT AND IMPORTANT 



Thursday 
April 24 
19 . 75 



Dear Colleague: 

As you no doubt know by now, ten consultants to the National Institute of 
^Education have been asked to review present funding policies for educational 
research and development, with particular references to policies that may have 
an impact upon regional educational laboratories and research and development 
centers. 

In order to be fully informed about the present state of educational 
research and development, the consultants are seeking information and ideas 
from a wide variety of sources. We held a face-to-face gathering on April 18 
in Chicago with leaders of laboratories and centers specifically, and we plan 
several on-site visits to learn more about laboratory and center activity first- 
hand. 

As Sam Sieber of our consultant group reviewed with the group of labora- 
tory and center people in Chicago, we feel a need for certain items of informa- 
tion from each organization individually. Therefore, we are sending you the 
enclosed ten questions for written reply. These will allow you to give us the 
most current program information and funding picture (the precise and full 
details of which may not be available at NIE), and will also allow you to express 
candidly your individual opinions and preferences about research and develop- 
ment funding policies. We appreciated the comiments many of you made in 
Chicago, and we have revised these questions again to take your advice into 
account. 
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The intent of this request is not to gain a totally comprehensive picture 
of your organization, but to learn about certain aspects in which the group is 
most interested and to give you a chance to highlight views you feel should be 
brought to our attention. We realize you could respond at any length to many 
of these items, with extensive and valuable information. Please do not 
misvmderstand our request for brief answers wherever possible; the consultants 
simply wish to give each response personal attention, and the time allowed us 
by the NIE Acting Director and the Council permits us to review only limited 
data from each of you. 

If you will return your responses to all questions except number 10 to my 
associate, Mr. P. Alistair Mac Kinnon, at the Washington Office of the New 
York State Education Department, 1201 l6th Street, N. W. , Box 100, Washington, 
D. C. 20036, we will be much obliged, I enclose a postpaid response envelope. 
For yotir response to number 10, I enclose a second envelope so that, as we agreed 
in Chicago, this information can pass directly to** Sam Sieber in confidence. A 
prompt reply, no later than Friday, May 9, 1975, will aid us in making use of 
your data. 

Please feel free to add any thoughts to those requested, or to write to us 
later if you wish* We appreciated your kind offers of help made in Chicago, and 
we thank you in advance for your assistance. 

With renewed thanks and best wishes, 

Faithfully yours. 



Ewald B. Nyqui 



Nyqui s t 
Oil Behalf of the Consultants 




Enclosures (3) 
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STUDY OF ALTERNATIVE FEDERAL RESEARCH AND DEVELOPMENT 
FUNDING POLICIES IN EDUCATION 



REQUEST FOR INFORMATION 
April 24, 1975 



1. Description of programs 

a. Please describe your major activities --programs or projects- -in 

educational research, development, evaluation, and dissemination. 
• We would like to have the following information for each major 

activity, displayed in the format shown on the attached sheet. 

o Short title 

o Description of the activity (limited to a sentence or two; please 

also classify the activity as to its major focus: research, 

development, evaluation, and dissemination) 
o Total amount of funds that will have been invested in the activity 

by the time of presently scheduled completion 
o Source(s) of funds for each activity (name agency or other source(s) 

and indicate type of funding arrangement, such as grant, contract, 

subcontract, etc. ; include also method of funding, such as 

competitive award, non- competitive, etc.) 
o Number of current professional employees in the activity (FTE) 
o Number of professional employees (FTE) estimated to be working 

on the activity at time of presently scheduled completion 
o Number of employees who will be unable to be employed elsewhere 

in your organization upon termination of this activity (based on 

presently known future funds) 

b. If your organization also engages in activities which are not directly 

related to educational research, development, evaluation, and dis- 
semination (for example, training, media production, or other 
strictly service activities), please briefly describe these activities 
and their proportion of your organization's budget. 

t- 

2. Budget summary 

a. Using whatever 12-month period is your organization's fiscal year, 

please list in one chart the major activities supported in the most 
recently completed fiscal year, with the total funds spent on each 
major activity in that year, and the percent of the total expense each 
activity represents. 

b. Please show in a second unified display each of the major sources of 

funds to your organization during the same year used in (a) above, 
with amount of dollars from each, and the percent of total contributed 
by each source. 
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c. If conditions during the present fiscal year are substantially different 
from the conditions presented in the display under (a) and (b^ above, 
please briefly explain the changes. 

Needed funds Beginning with your next fiscal year, what NIE funds do you 
feel you realistically need in order to pursue the goals of your organiza- 
tion? 

Search for other funds Are you pursuing any other source(s) of support 
beyond the NIE? If so, could you tell us which? What influence has this 
search for funds had on other activity in your organization? Please be as 
specific as you can on each. 

Sources of future funds What is your best estimate of your funding mix in 
your own fiscal year 1976, from all sources? Please display a projected 
set of sources, in a format like that in question Z(b). 

Desired continuations Which projects or major lines of work in your organ- 
ization are most important to continue beyond the current year, and why? 

Relations with users and outside sources We feel it is important to under- 
stand how you coordinate your work with the perceptions and policies of 
others such as state and local education agencies, community groups, 
other laboratories and/or centers, scholars beyond your own staff, and 
other professionals. 

a. How do groups intended to be eventual users of your products or services 

influence the development of the programs and projects undertaken by 
your organization? Has the degree of quality of this influence changed 
in the past few years? Please indicate concretely the impact of such 
influence. 

b. During the past year, who was formally consulted, other than your own 

staff or board/chief academic administrator, on matters of policy such 
as beginning a new program or reviewing an operating program? 
Please include a list of actual consultants used. 

Help to others Beyond your direct programmatic activity in research and 
product development intended for future benefit to schools and education, 
how do you view your organization as a* source of substantial advice or 
technical assistance to people in other organizations on problems related 
to educational research, development, evaluation, or dissemination? 

a. Please discuss the role of your organization, at present, as a resource 
for people c.t the state and local education agency level, or in schools, 
and indicate the level of effort devoted to this activity, if any. 
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b. Please discuss the role of your organization as a resource for research 
and development personnel in other research organizations or univer- 
sities, and indicate the level of effort devoted to this activity, if any. 

9. Rationale for institutional support What special features of your organization 
appear to you to qualify it for ^'institutional support^' from the government? 
What specific form do you feel that support should take, and why? 

10. ( NOTE ; Put your answer to this question on separate sheets, with the name 
of the responding organization, and send it to Sam Sieber at his home 
address, as detailed in the covering letter accompanying this list of ques- 
tions. ) 

Relations with NIE What are the major strengths and weaknesses in the 
relationship between your organization and NIE over the past two to three 
years? (The more specific you can make your response, the more helpful 
it will be. ) 



Please feel free to add any facts or views on these, or related, matters 
which you feel the consultants should take into account. 



Atta chments 

1. Format for table for program information, Cuestion 1. 

2. Format for budget summary and fund-source summary, Cuestion 2, and 

source of future funds. Question 5. 
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REGIONAL EDUCATIONAL LABOPwATORlES 
FUNDING AND STAFFING DATA^ 
APRIL 19/5 



Appalachian Education Lab 
Charleston, West Virginia 

Central Mid-western Educa- 
tional' Regional Lab 
St Louis, Kissouci 

Far West Laboratory 

San Francisco » California 

Northwest Regional Educa- 
tional Laboratory 
Portland, Oregon 

Research for Better Schools 
Philadelphia, Pennsylvania 

Southwest Educational 
Development Lab 
Austin, Texas 

Southwest Regional Lab 
Los Alamitos, California 



TOTAL 



197A 



Total 
Budget 



2,013 
3,178 

5,079 
6,171 

4,575 
3,310 



27,226 



NIE 
% 



96% 



79% 



62% 



68% 



95% 



56% 



2,900 iOO% 



1975 

Total NIE 
Budget kmt. 



NIE 
% 



1,695 
2,323 

6,289 
6,056 

3,500 
3,183 



1,213 
1,839 

3,16A 
3,809 

3,325 
1,850 



91% 



70*/ 



50% 



63% 



95% 



58% 



3,060 3,060 100% 



26,106 18,260 



1976 

Funds Desired 
from NIE^ 

3,100 
1,900 



4,656 

A, 600 
2,000 

3,060 

19,316 



FTE Professional Staff 



Total 



36 



79 



129 



106 



129 



97 



85 



661 



NIE 
Funded 



36 



47 



64 



74 



129 



62 



84 



496 



NIE _ 
Marginal-^ 



22 
40 

45 
59 

122 
54 

84 

426 



^ Source: questionnaire distributed by consultants . All dollar amourits in thousands. 

2 This figure was given in response to the question: "Beginning with your next fiscal year, what NIE funds do you feel you realis- 
tically need in order to pursue the goals of your organization?" ^ , , ^ » , 

3'N^umber of employees who will be unable to be employed elsewhere ir. 'zw^ organization upon termination of their present activity 
(based on presently known future funds)." 
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1974 



Total NIE 
Budget % 

Center for Educational 1,305 937* 

Policy & Management 
University of Oregon 

Learning R&D Center 3,581 82% 

University of Pittsburgh 

Center for the Social 589 96% 

Organization of Schools 
John Hopkins University 



Center for Study of Evaluation 1,294 
University of CA., Los Angeles 



82% 



Center for Vocational Educ. 3,880 69% 

Ohio State University 

National Center for Higher 1,959 91% 

Education Management Systeins 
Western Interstate Coram, for Higher 
Education 

R&D Center for Teacher Educ. 818 86% 

University' of Texas 

Stanford Center for R&D in 1,028 99% 

Teaching 
Stanford University 

Wisconsin R&D Center 3,087 91% 

University of Wisconsin 

R&D Centers TOTAL 15,892 
Regional Laboratories TOTAL 27,226 
GRAND TOTAL 43,118 

^ ^ ^ Same notes apply as in previous table 
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RESEARCH AND DEVELOPMENT CENTERS 
FUNDING AND STAFFING DATA 
APRIL 1975 



1975 



1976 FTE Professional Staff 



Total NIE NIE Funds Desired NIE NIE 

Budget Amt. % from NIE^ Total Fund ed Marginal; 

1,505 1,399 93% 1,133 32 32 29 



3,081 2,137 69% 2,600* 124 124 0 

518 492 95% 450 12 12 9 

1,268 1.154 90% 1,072 29 27 27 

3,907 2,344 . 60% 3,500 89 50 43 

1,610 1.557 97% 2,650 ' 28 28 28 

910 850 -93% • 1,194 33 33 33 

1,028 1,016 99% 1,565 32 32 31 

3,411 2,698 78% 3,644 i2? IZ 



15,628 * 12,090 15,158 453 426 189 
26,106 18,260 19,316 661 jm 426 
347474 1,H4 922 615 



41,734 30,350 
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Appendix C (2) 



Analysis of Responses to Q. 10 by Lab and Center Directors: 
"What are the major strengths and weaknesses in the relation- 
ship between your organization and NIE over the past two or 
three years?" 

This analysis is based on the following responses: 



Center directors = 8 
Lab directors = 5 

1. Although the question was worded to permit the directors to 
mention both positive and negative aspects of their relation- 
ship with NIE, the negative responses far outnumbered the 
positive ones. Only 4 of the Centers mentioned a total of 7 
"strengths," as d^. only 3 of the Labs. As one director noted: 
"When this question was posed to our key staff, they concluded 
that they had great difficulty in defining any major strengths 
in the relationship between (the Lab) and NIE." All of the 
Centers and Labs mentioned "weaknesses," however — a total 
of 36 by the Centers and 34 by the Labs. 

2: As shown in the above figures, the Labs and Centers were equally 

critical of NIE. Further, with regard to only two negative aspects 
were there somewhat more comments from one type of agency: 4 Labs 
but only 2 Centers mentioned personnel turnover as a problem; and 
3 Lab*s and only 1 Center mentioned a negati^^e bias on the part of 
NIE towards these agencies. Otherwise the. comments of Lab and 
Center directors were virtually indistinguishable. Since there 
was little differentiation, our tabulations below will not 
distinguish these two kinds of agencies. N 

Directors 

3. The "strengths" mentioned by the directors were the following: 
Competence, commitment, help of NIE staff 

Specific persons, e.g., project officer, 
contracts officer, director or associate 
director ^ 

Specific group, e.g., pre-NIE staff, cer- 
tain programs 

In general _Ji 

8 

Review, monitoring process 

In beginning of NIE ^ 



4 



Recent improvement 

Numerous staff members have visited 
agency 



1 



2 
3 

Long-term contract ^ 



Recent evidence that NIE wishes to relate 
Q to sea's 
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Description of '^strengths*' in the relationship 

More than half of the directors referred to the helpfulness 
and competence of NIE personnel . It is noteworthy, however, 
that they singled out particular persons rather than the staff 
as a whole. As one Lab director said, "The major strengths in 
(the Lab's) relationship with NIE are generally attributable to 
individual NIE staff members.** With an enthusiasm that was 
typical of several directors, one mentioned two individuals in 
glowing terms: 

X, who has been our project officer for the last two 
years, has proved to be a highly intelligent, knowledge- 
able, open-minded and decisive person. She is profes- 
sionally dedicated and is thoroughly honest about 
recognizing facts when facts are presented to her. ... 
The second bright spot has been the presence and active 
leadership of Y. We are extremely unhappy that he is 
probably going to be leaving NIE within the next month or 
two. He is the one professionally trained person we have 
encountered within NIE middle-Tiianagement who has both a 
substantive grasp of the field in which we work and a clear, 
purposeful vision of the kinds and quality of work that need 
to be done to solve some of the highest priority problems in 
American education. 

This tendency to focus on individuals would seem to reflect an 
inability to cite virtues in NIE as an agency. Indeed, several 
directors were quite explicit about this point. One mentioned 
the valuable "colleaguial relationships" with NIE staff which 
were being hampered by "bureaucratic rules." Another summed 
up the point as follows: 

NIE to date has been less than the sum of its parts. 
Our relations with individuals within the Institute 
have been cordial and at a high professional level. 
But by the time these come up out through the top they 
bear little resemblance to the interchange at the 
individual and unit level. ... For reasons "beyond 
their control" no one within th^ agency has been able 
to formulate and hold a position that is consistent 
with what has been put forth as a "commitment" at an 
earlier time. 

Nevertheless, the tendency of the directors to laud individual 
staff members reveals that the agency has a strong professional 
group to build upon. That organizational problems might hamper 
"he work of these individuals will be suggested later when we 
turn to the major "weaknesses" in the relationship with NIE. 
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The only other strength that was mentioned by as many as 3 directors 
concerned the review/monitoring process . However, these comments 
were highly qualified. One director referred only to the beginning 
of NIE, and another said that the review process had improved some- 
what in the recent past. Two noted that many staff members of NIE 
had visited their organizations over the years. 



5. The "weaknesses" mentioned by the directors were the following: 
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Review, monitoring process 
Quantity 

Too frequent, burdensome, costly 
Quality 

Visitors not competent to judge 

Lack of consistent priorities from NIE 

Poor standards 

Lack of monitoring in recent years 
Last minute, crisis-oriented reviews 
Need to compare products of labs-centers 
vs. others 

Discontinuity in policy, plans, procedures, priorities 
Communications 

Lack of involvement of field, unresponsive to field 

Lack of communication to field about decision-making 
process; top clique makes decisions in closed 
manner; arbitrary decisions imposed on managers- 
Delays in reaching decisions, negotiating contracts , 
etc. 

Poor internal communication in NIE, conflict among 
staff 

Personnel turnover 

Procurement 

Dealing with individual projects, programs 

Competitive basis 

No provision for start-ups 

No funds for field-initiated research 



N 

directors 



10 



1 
2 
2 
3 
1 



13 
9 



6 
6 



11 
6 



4 
3 
2 
1 
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Negative bias toward centers-labs, hostility 4 

Incompetence of staff (lack of training, experience) 3 

Clearance of research forms 2 

Political ineptitude ^ 

Dissemination guidelines too restrictive 1 

Non-contract demands (for product descriptions, etc.) 1 

R&D model (linear, emphasizes research) 1 

F ailure to recognize distinction between labs and centers 1 

Exclusion of postsecondary education from plans , 

especially regarding dissemination 1 
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Visits by consultants 

!• Regional educational laboratories 

Far West Laboratory for Educational Research and Development 
San Francisco, California 

Southwest Educational Development Laboratory 
Austin, Texas 

Southwest Regional Laboratory for Educational Research and 
Development 

Los Alamitos, California 

Central Midwestern Regional Educational Laboratory, Inc. (CEMREL) 
St. Louis, Missouri 



2. Research and development centers 

Research and Development Center for Teacher Education 
University of Texas, Austin 

Stanford Center for Research and Development in Teaching 
Stanford University 

Wisconsin Research and Development Center for Cognitive Learning 
University of Wisconsin, Madison 



3. Other R&D performers 

Center for Research and Development in Higher Education 
University of California, Berkeley 

Education Development Center, Inc. 
Newton, Massachusetts 

Stanford Research Institute 
Menlo Park, California 

Educational Testing Service 
Princeton, New Jersey 
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Personal meetings 

American Association of Colleges of Teacher Education (Executive Director) 
American Association of School Administrators (President-elect) 
American Educational Research Association (President, Executive Director) 
Learning Research and Development Center, University of 

Pittsburgh (Director) 
National Education Association (group of staff) 

National Science Foundation, Education Directorate (group of staff) 

Research for Better Schools, Inc. (Director) 

U.S. House of Representatives, Committee on Appropriations, 

Subcommittee on Departments of Labor and HEW (staff) 
U.S. Senate, Committee on Appropriations, Subcommittee on Departments 

of Labor and HEW (staff) 

Teachers College, Columbia University (President) 

Harvard University Graduate School of Education (Dean and group of faculty) 



Correspondence received from individuals 

Stephen K. Bailey 

American Council on Education 

George Brain 

Washington State University 
R. Louis Bright 

Western Institute for Science and Technology 
John Flanagan 

American Institutes for Research 

J. We Getzels 
University of Chicago 

Keith Goldhammer 
Michigan State University 

Samuel Halperin 

Institute for Educational Leadership 
Kenneth' Hansen 

Nevada State Department of education 

Lawrence Haskew 
University of Texas 

Richard Herlig 

Kansas State .Department of Education 
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Francis lanni 

Horace Mann-Lincoln Institute, Teachers College Columbia University 

Francis Keppel , 
Aspen Institute for Humanistic Studies. 

Kenneth Lindsay 

Utah State Board of Education 

Sidney Marland 

College Entrance Examination Board 
Ward S. Mason 

National Institute of Education 
Frank Mattas 

Educational Resources Center 
Charles Mojkowski 

Rhode Island State Department of Education 
Cary Potter 

National Association of Independent Schools 
David Robinson 

Carnegie Corporation of New York 
Judith Segal 

National Institute of Education ^ 

Ralph Tyler \ . / 

Science Research Associates . * 

Theodore Waller . 
Grolier Educational Corporation 

Sheldon White 
Harvard University 
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